Optimal robust formation control for heterogeneous multi-agent systems based on reinforcement learning

In this article, a reinforcement learning (RL)-based robust control strategy is proposed for uncertain heterogeneous multi-agent systems to achieve optimal collision-free time-varying formations. Without using any global information, a fully distributed adaptive observer is developed to estimate both dynamics and states of the reference and disturbance systems. The observer parameters are found by an observed model-based or a model-free off-policy RL algorithm. Using the internal model principle, a novel optimal robust formation control strategy is developed based on another proposed off-policy RL algorithm. The algorithm addresses the nonquadratic optimization problem when the system model is completely unknown. Taking the bushfire edge tracking and patrolling task for an unmanned aerial vehicle-unmanned ground vehicle heterogeneous system as an example, the effectiveness and robustness of the developed control strategy are verified by simulations.