Energy Efficient Transmission in Underlay CR-NOMA Networks Enabled by Reinforcement Learning

In order to improve the energy efficiency (EE) in the underlay cognitive radio (CR)networks, a power allocation strategy based on an actor-critic reinforcement learning is proposed, where a cluster of cognitive users (CUs) can simultaneously access to the same primary spectrum band under the interference constraints of the primary user (PU), by employing the non-orthogonal multiple access (NOMA) technique. In the proposed scheme, the optimization of the power allocation is formulated as a non-convex optimization problem. Additionally, the power allocation for different CUs is based on the actor-critic reinforcement learning model, in which the weighted data rate is set as the reward function,and the generated action strategy (i.e. the power allocation) is iteratively criticized and updated. Both the CU's spectral efficiency and the PU's interference constrains are considered in the training of the actor-critic reinforcement learning. Furthermore, the first order Taylor approximation as well as other manipulations are adopted to solve the power allocation optimization problem for the sake of considering the conventional channel conditions. According to the simulation results, we find that our scheme could achieve a higher spectral efficiency for the CUs compared to a benchmark scheme without learning process as well as the existing Q-learning based method, while the resultant interference affecting the PU transmission can be maintained at a given tolerated limit.