Joint Topology Construction and Power Adjustment for UAV Networks: A Deep Reinforcement Learning Based Approach

In this paper, we investigate a backhaul framework jointly considering topology construction and power adjustment for self-organizing UAV networks. To enhance the backhaul rate with limited information exchange and avoid malicious power competition, we propose a deep reinforcement learning(DRL) based method to construct the backhaul framework where each UAV distributedly makes decisions. First, we decompose the backhaul framework into three submodules, i.e., transmission target selection(TS), total power control(PC), and multi-channel power allocation(PA). Then, the three submodules are solved by heterogeneous DRL algorithms with tailored rewards to regulate UAVs’ behaviors. In particular, TS is solved by deep-Q learning to construct topology with less relay and guarantee the backhaul rate.PC and PA are solved by deep deterministic policy gradient to match the traffic requirement with proper finegrained transmission power. As a result, the malicious power competition is alleviated, and the backhaul rate is further enhanced. Simulation results show that the proposed framework effectively achieves system-level and all-around performance gain compared with DQL and max-min method, i.e., higher backhaul rate, lower transmission power, and fewer hop.