Improved reinforcement learning in cooperative multi-agent environments using knowledge transfer

Nowadays, cooperative multi-agent systems are used to learn how to achieve goals in large-scale dynamic environments. However, learning in these environments is challenging: from the effect of search space size on learning time to inefficient cooperation among agents. Moreover, reinforcement learning algorithms may suffer from a long time of convergence in such environments. In this paper, a communication framework is introduced. In the proposed communication framework, agents learn to cooperate effectively, and also by the introduction of a new state calculation method, the size of state space has declined considerably. Furthermore, a knowledge transferring algorithm is presented to share the gained experiences among the different agents, and an effective knowledge fusing mechanism is developed to fuse the agents' own experiences with the experiences received from other team members. Finally, the simulation results are provided to indicate the effectiveness of the proposed method in complex learning tasks. We have evaluated our approach on the shepherding problem and the results show that the learning process has been accelerated by making use of the knowledge transferring mechanism and the size of state space has been declined by generating similar states based on the state abstraction concept.