Sarsa-based Trajectory Planning of Multi-UAVs in Dense Mesh Router Networks

Deploying wireless routers on the ground and un-manned aerial vehicles (UAVs) in the air is believed to be a fast and efficient approach to providing the emergency communication service to disaster areas. The network lifetime is restricted to the lifetime of mesh networks of routers that are scattered across a complex disaster environment. We consider the problem of multi-UAVs relaying and moving with the goal of maximizing the network lifetime. UAVs must learn where to move in order to relay messages from the routers efficiently. However, under the dynamics of the router traffic, that is, the uncertainty of the environment, it is challenging for UAVs to find a movement mechanism that maximizes the network lifetime. By embracing the on-policy reinforcement learning algorithm Sarsa, we are able to demonstrate a greedy movement policy. Specifically, we study the trajectory planning of multiple UAVs in a dense wireless router mesh networks (WMNs) on the ground. UAVs can learn the unknown environment by a little movement of UAVs in each step, in which communication connections between routers with UAVs are retained. A Q-table of movement actions and environment states is formed after multiple attempts of movements. Simulation results show the proposal effectiveness comparing with other methods.