Leveraging Expert Demonstrations in Robot Cooperation with Multi-Agent Reinforcement Learning

While deep reinforcement learning (DRL) enhances the flexibility and intelligence of a single robot, it has proven challenging to solve the cooperatively of even basic tasks. And robotic manipulation is cumbersome and can easily yield getting trapped in local optima with reward shaping. As such sparse rewards are an attractive alternative. In this paper, we demonstrate how teams of robots are able to solve cooperative tasks. Additionally, we provide insights on how to facilitate exploration and faster learning in collaborative systems. First, we increased the amount of effective data samples in the replay buffer by leveraging virtual targets. Secondly, we introduce a small number of expert demonstrations to guide the robot during training via an additional loss that forces the policy network to learn the expert data faster. Finally, to improve the quality of behavior cloning, we propose a Judge mechanism that updates the strategy by selecting optimal action while training. Furthermore, our algorithms were tested in simulation using both dual arms and teams of two robots with single arms.