Energy-efficient power control strategy of the delay tolerable service based on the reinforcement learning

In recent years, the rapid development of Internet technology and its applications has led to an exponential growth in the number of Internet users and wireless terminal devices, resulting in a corresponding increase in energy consumption. This has necessitated the need to reduce energy consumption while maintaining the quality of communication services. To this end, we investigate the possibility of improving energy efficiency (EE) of delay tolerable (DT) services by allocating resources based on the time-domain water-filling algorithm. We first transform the non-convex problem of maximizing EE into a convex problem of minimizing transmission power to obtain the optimal solution, and then use the greedy algorithm to obtain an upper bound. Furthermore, to capture a more realistic scenario, an Approximate Statistical Dynamic Programming (ASDP) algorithm is introduced, but its effect on enhancing EE is limited. To overcome this limitation, three Deep Reinforcement Learning (DRL) algorithms are implemented. The simulations results show that the settings of maximum transmit power and SNR during agent training have an impact on the performance of the agent. Finally, by comparing the mean values of transmission power, outage probability, equilibrium power and performance improvement percentage of several algorithms, we conclude that the Deep Deterministic Policy Gradient (DDPG) algorithm produces the best agent performance in the environment with a fixed SNR of 2 (dB).