Deep Q-Learning based Dynamic Resource Allocation for Self-Powered Ultra-Dense Networks

Though enhancing the capacity and coverage of cellular networks to meet the explosive increasing of traffic demands, Ultra-Dense Network (UDN) suffers from great power consumption and greenhouse gas emission. Turning small base stations (SBS) off dynamically according to the network operating conditions is a promising solution to enhance energy efficiency (EE). Inspired by the success of Deep Q-learning Network(DQN) on solving complicated real-time control problems and the usage of energy-harvesting(EH) in UDNs, without any prior knowledge about energy arrival, data arrival and channel state information, we present a DQN-based framework for dynamic resource allocation in EH-UDN. Specially, joint optimization problem is formulated, considering both EE and quality of service (QoS) of the network. Then according to the application scenario, the action space, state space and reward function of the proposed DQN-based framework are defined and formulated. We evaluate the performance of the proposed framework by comparing it with the classic Q-learning framework via simulation. Numerical results show that our proposed scheme can enhance EE while taking good control of the QoS.