An approach to the pursuit problem on a heterogeneous multiagent system using reinforcement learning

Cooperation among agents is important for multiagent systems having a shared goal. In this paper, an example of the pursuit problem is studied, in which four hunters collaborate to catch a target. A reinforcement learning algorithm is employed to model how the hunters acquire this cooperative behavior to achieve the task. In order to apply Q-learning, which is one way of reinforcement learning, two kinds of prediction are needed for each hunter agent. One is the location of the other hunter agents and target agent, and the other is the movement direction of the target agent at next time step t. In our treatment we extend the standard problem to systems with heterogeneous agents. One motivation for this is that the target agent and hunter agents have differing abilities. In addition, even though those hunter agents are homogeneous at the beginning of the problem, their abilities become heterogeneous in the learning process. Simulations of this pursuit problem were performed on a continuous action state space, the results of which are displayed, accompanied by a discussion of their outcomes' dependence upon the initial locations of the hunters and the speeds of the hunters and a target. (C) 2003 Elsevier Science B.V. All rights reserved.