Joint Power Allocation and User Fairness Optimization for Reinforcement Learning Over mmWave-NOMA Heterogeneous Networks

In this paper, the problem of joint power allocation and user fairness is investigated for an mmWave heterogeneous network (HetNet) including hybrid non-orthogonal multiple access (NOMA) and orthogonal multiple access (OMA) transmissions. In particular, in the assumed realistic model, the uplink (UL) of macrocell users (MCUs) and the downlink (DL) of small cell users (SCUs) share the same resource block (RB) to increase the network capacity. Furthermore, an imperfect successive interference cancelation (SIC) decoding is considered due to hardware impairment of real-world NOMA systems. Based on the number of RBs and clusters, we consider two cases as fully resource allocation and partially resource allocation. The multi-objective optimization problem (MOOP), i.e., user fairness maximization and transmission power minimization is transformed into single objective optimization problem (SOOP) by weighted sum (WS) and $\varepsilon$-constraint (EC) methods. Several types of reinforcement learning (RL) such as Q-learning (QL), deep Q-learning network (DQN), and deep deterministic policy gradient (DDPG) are employed to solve the optimization problems subject to the minimum quality of service (QoS), minimum effect on the OMA users, imperfect SIC, and RB allocation constraints. The results indicate the efficiency of the proposed methods.