Experimental Evaluation of a Method for Simulation based Learning for a Multi-Agent System Acting in a Physical Environment

A method for simulation based reinforcement learning (RL) for a multi-agent system acting in a physical environment is introduced, which is based on Multi-Agent Actor-Critic (MAAC) reinforcement learning. In the proposed method, avatar agents learn in a simulated model of the physical environment and the learned experience is then used by agents in the actual physical environment. The proposed concept is verified using a laboratory benchmark setup in which multiple agents, acting within the same environment, are required to coordinate their movement actions to prevent collisions. Three state-of-the-art algorithms for multi-agent reinforcement learning (MARL) are evaluated, with respect to their applicability for a predefined benchmark scenario. Based on simulations it is shown that the MAAC method is most applicable for implementation as it provides effective distributed learning and suits well to the concept of learning in simulated environments. Our experimental results, which compare simulated learning and task execution in a simulated environment with that of task execution in a physical environment demonstrate the feasibility of the proposed concept.