DRL-based Distributed Reconfigurable Intelligent Surfaces-Aided mmWave Communications

Reconfigurable intelligent surfaces (RIS) technology is a promising technique with the potential to surpass massive multiple input multiple output (massive-MIMO) technology, and offer higher performance and broader application prospects for future wireless communications. In this work, We have developed a method based on distributed deep reinforcement learning (DRL) to jointly optimize the phase shift matrix of RIS and the beamforming matrix of base stations, maximizing the throughput of the considered system. We have proposed a distributed downlink beamforming method, utilizing the Distributed Deterministic Policy Gradient with prioritized experience replay(PRE-D4PG) algorithm to train DRL models. The convergence and performance of the system are enhanced by adopting a prioritized experience replay mechanism, collecting distributed experiences from different RIS, the convergence and performance of the system are enhanced. Subsequently, we bench marked the proposed solution against two commonly used linear beamforming schemes. The results indicate that the PRE-D4PG scheme with distributed experiences achieves optimal performance in terms of network scalability.