Multi-robot box-pushing: Single-agent Q-learning vs. team Q-learning

In this paper, two types of multi-agent reinforcement learning algorithms are employed in a task of multi-robot bog-pushing. The first one is a direct extension of the single-agent Q-learning, which does not have a solid theoretical foundation because it violates the static environment assumption of the Q-learning algorithm. The second one is the Team Q-learning algorithm, which is a multi-agent reinforcement learning algorithm, and is proved to converge to the optimal policy. The states, actions, and reward function of the algorithms are presented in the paper. Based on the two Q-learning algorithms, a fully distributed multi-robot system is developed. Computer simulations are carried out using the developed system. The simulation results show that the two algorithms are effective in a simple environment. It is shown, however, that the single-agent Q-learning algorithm does a better job than the Team Q-learning algorithm in a complicated and unknown environment with many obstacles.