Multi-Agent Model-Based Reinforcement Learning for Trajectory Design and Power Control in UAV-Enabled Networks

Unmanned aerial vehicles (UAVs) serving as aerial base stations is a promising technology for wireless communications. This paper formulates a joint optimization problem of UAV trajectory design and power control to minimize the power consumption when satisfying users' QoS requirements in a downlink transmission. Firstly, a multi-agent deep deterministic policy gradient (MADDPG) scheme with centralized training and decentralized execution is proposed to improve the overall performance of the UAVs in cooperation. Secondly, model value expansion (MVE) is incorporated into the model-free MADDPG scheme. By imaging future transitions, the proposed multi-agent model value expansion deep deterministic policy gradient (MA-MVE-DDPG) algorithm generates more experiences, and thus accelerates training. Simulation results have demonstrated that our proposed MA-MVE-DDPG algorithm achieves better performance and converges faster than benchmark schemes.