Formation Tracking of Spatiotemporal Multiagent Systems: A Decentralized Reinforcement Learning Approach

This article investigates the formation tracking problem for discrete-time uncertain spatiotemporal multiagent systems (MASs). Note that the common multiagent reinforcement learning (MARL) method requires the actions and states of all agents to train the centralized critic; hence, this method may be impractical in constrained communication. Therefore, a decentralized RL framework is proposed that combines a neural network boundary approximation distributed observer (NNBADO) and an intelligent nonaffine leader (INL). As a result, the formation tracking problem for each agent can be modeled as a partially observable Markov decision process (POMDP). A novel RL formation tracking algorithm is designed based on a fusion reward scheme synthesizing the orbit tracking and formation objectives. The experiment results show that our algorithm can improve the formation accuracy.