Reinforcement Learning Based Dual-Functional Massive MIMO Systems for Multi-Target Detection and Communications

Reinforcement learning (RL) based approaches in massive multiple input multiple output (mMIMO) arrays allow target detection in unknown environments. However, there are two main drawbacks hindering the practical application of these approaches: (i) poor detection performance for weak targets, and (ii) mismatch between high system overhead and single functionality. In light of this, we propose a dual-functional mMIMO (DF-mMIMO) system for multi-target detection with embedded communication in this work. First, we improve the RL based multi-target detection algorithm in both "action" and "reward" steps, by adding an omni-directional detection pulse in the action step and optimizing the reward mechanism in the reward step, so greatly improving the detection probability of weak targets in strong clutter. To achieve the dual modalities, communication information is embedded into the radar transmit waveform via complex beampattern modulation. In particular, we propose a low computational complexity two-step beamformer design method. First, the transmit waveform covariance matrix is designed via convex optimization, and then the beamforming weight matrix is determined according to closed-form formulas. Extensive simulation results demonstrate that the proposed DF-mMIMO system exhibits excellent target detection capability in a scenario where both strong and weak targets co-exist with downlink communications.