Intelligent Beam Management Based on Deep Reinforcement Learning in High-Speed Railway Scenarios

Millimeter-wave (mm-wave) communications can fundamentally solve the problem of spectrum shortage in wireless communication systems, and many progresses have been made in standardization, which laid the foundation for the application of mm-wave in high-speed railway (HSR) scenarios. However, the HSR channel is fast time-varying and difficult to model. Also beamforming is essential to improve the directional gain of the antenna and offset the high path loss of mm-wave. But the high-speed movement of train makes the beam management extremely challenging, and the trade-off between achievable performance and beam training overhead is unavoidable. Reinforcement learning (RL) can offer new solutions to this problem, as it does not need a large number of training samples and other system information, and is capable of achieving high performance with low complexity. In this article, we propose an intelligent beam management scheme based on a deep RL algorithm called deep Q-network (DQN), and our main idea is to exploit the hidden patterns of mm-wave train-ground communication system to improve the downlink signal-to-noise ratio (SNR), while ensuring a certain communication stability and imposing a minimal training overhead. Through extensive simulations, we demonstrate that the proposed DQN-based scheme has better performance than the four baseline schemes, and it also offers great advantages in SNR stability and implementation complexity.