Throughput Maximization by Deep Reinforcement Learning With Energy Cooperation for Renewable Ultradense IoT Networks

Ultradense network (UDN) is considered as one of the key technologies for the explosive growth of mobile traffic demand on the Internet of Things (IoT). It enhances network capacity by deploying small base stations in large quantities, but it simultaneously causes great energy consumption. In this article, we use energy harvesting (EH) and energy cooperation technologies to maximize system throughput and save energy. Considering that the energy arrival process and channel information are not available a priori, we propose an optimal deep reinforcement learning (DRL) algorithm to solve this average throughput maximization problem over a finite horizon. We also propose a multiagent DRL method to solve the dimensionality problem caused by the expansion of the state and action dimensions. Finally, we compare these algorithms with two traditional algorithms, greedy algorithm and conservative algorithm. The numerical results show that the proposed algorithms are valid and effective in increasing system average throughput on the long term.