Energy-efficient UAV-enabled computation offloading for industrial internet of things: a deep reinforcement learning approach

Industrial Internet of Things (IIoT) has been envisioned as a killer application of 5G and beyond. However, due to the shortness of computation ablility and batery capacity, it is challenging for IIoT devices to process latency-sensitive and resource-sensitive tasks. Moblie Edge Computing (MEC), as a promising paradigm for handling tasks with high quality of service (QoS) requirement and for energy-constrained IIoT devices, allows IIoT devices to offload their tasks to MEC servers, which can significantly reduce the task process delay and energy consumptions. However, the deployment of the MEC servers rely heavily on communication infrastructure, which greatly reduce the flexibility. Toward this end, in this paper, we consider multiple Unmanned Aerial Vehicles (UAV) eqqipped with transceivers as aerial MEC servers to provide IIoT devices computation offloading opportunities due to their high controbility. IIoT devices can choose to offload the tasks to UAVs through air-ground links, or to offload the tasks to the remote cloud center through ground cellular network, or to process the tasks locally. We formulate the multi-UAV-Enabled computation offloading problem as a mixed integer non-linear programming (MINLP) problem and prove its NP-hardness. To obtain the energy-efficient and low complexity solution, we propose an intelligent computation offloading algorithm called multi-agent deep Q-learning with stochastic prioritized replay (MDSPR). Numerical results show that the proposed MDSPR converges fast and outperforms the benchmark algorithms, including random method, deep Q-learning method and double deep Q-learning method in terms of energy efficiency and task successful rate.