Trajectory Design for Overlay UAV-to-Device Communications by Deep Reinforcement Learning

In this paper, we consider a cellular Internet of unmanned aerial vehicles (UAVs) where the sensory data can be transmitted to the mobile devices directly by overlaying UAV-to-Device (U2D) communications, or through the base station (BS) by cellular communications. Since the transmission modes of UAVs may influence their trajectories, we study the trajectory design problem for UAVs aiming to maximize the total utility in consideration of their transmission modes. This problem is a Markov decision problem (MDP) with a large state-action space, and thus, we propose a multi-UAV trajectory design algorithm using multi-agent deep reinforcement learning (DRL) to solve this problem. Simulation results show that our proposed algorithm can achieve a higher total utility than the single-agent method.