User-centric AP Clustering with Deep Reinforcement Learning for Cell-Free Massive MIMO

Cell-free massive MIMO (CF-mMIMO), which performs centralized MIMO processing for densely distributed access points (APs), has been proposed for future mobile networks. In CF-mMIMO, user equipments (UEs) are served by a limited number of APs to reduce the computational load for signal processing while meeting throughput requirements. AP clustering, which is selecting APs for each UE, in dynamic environments that UEs move in is the main challenge. To find optimal AP clustering, one approach is to use deep reinforcement learning (DRL). However, the number of possible actions in DRL increases with the number of APs and Ues. The issue of scalability due to the increasing size of the neural network (NN) in a large-scale environment remains a problem. To address this problem, we propose user-centric AP clustering with distributed-DRL which has actors for each UE, and an action design which specifies the difference in AP cluster size from the previous time step. In the proposed method, since the number of possible actions does not depend on the number of APs and UEs, the size of NN can be reduced. The numerical simulation shows the proposed method achieves AP clustering that reduces the computational load while meeting the throughput requirements, with high scalability in terms of the number of UEs and APs.