Deep Reinforcement Learning Based Latency Minimization for Mobile Edge Computing With Virtualization in Maritime UAV Communication Network

The rapid development of maritime activities has led to the emergence of more and more computation-intensive applications. In order to meet the huge demand for wireless communications in maritime environment, mobile edge computing (MEC) is considered as an effective solution to provide powerful computing capabilities for maritime terminals of resource scarcity or latency sensitive. A basic technology to implement MEC is virtual machine (VM) multiplexing, through which multi-task parallel computing on a server is realized. In this paper, a two-layer unmanned aerial vehicles (UAVs) maritime communication network with a centralized top-UAV (T-UAV) and a group of distributed bottom-UAVs (B-UAVs) is established and MEC is used on T-UAV. We aim to solve the latency minimization problem for both communication and computation in this maritime UAV swarm mobile edge computing network. We reformulate this problem into a Markov decision process (MDP), since it is a non-convex and multiply constrained but has the characteristics of MDP. Based on this MDP model, we take deep reinforcement learning (DRL) as our tool to propose a deep Q-network (DQN) and a deep deterministic policy gradient (DDPG) algorithms to optimize the trajectory of T-UAV and configuration of virtual machines (VMs). Using these two proposed algorithms, we can minimize the system latency. Simulation results show that the given solutions are valid and effective.