Air-M: A Visual Reality Many-agent Reinforcement Learning Platform for Large-Scale Aerial Unmanned System

Reinforcement learning for swarms of flying robots is a challenging task that requires a large number of data samples. Moreover, the problem of sim-to-real transfer has long been a challenge in robotics algorithm deployment. To address these issues, we propose Air-M, a platform that facilitates large-scale drone swarm learning in a distributed docker container environment and deployment in a virtual reality setting. Air-M trains the policy network using physics engines and creates replicas of agents in docker containers, which helps amortize the computational cost. In addition, Air-M establishes an intermediate link between the simulation and the real world, allowing real drones to interact with virtual objects via virtual sensors. This enables the policy network to be trained using virtual agents and seamlessly transferred to real drones. Air-M is highly scalable, accommodating hundreds of agents with dynamic models and virtual sensors. We evaluate the effectiveness of our approach by conducting experiments in three representative virtual scenarios with an increasing number of agents. Our results demonstrate that our method outperforms the state-of-the-art in terms of training efficiency and transferability, making it a promising platform for swarm robotics applications.