Learning-Assisted Secure Relay Selection with Outdated CSI for Finite-State Markov Channel

In this paper, we investigate secure relay selection for finite-state Markov channel and propose a Q-learning assisted relay selection scheme. Specifically, we firstly analyze the achievable effective secrecy throughput of random selection scheme and optimal selection scheme, respectively, showing that the secrecy performance is highly determined by relay selection methodology. Then, we leverage the Q-learning to learn how to select relay for finite-state Markov channel, which is capable of selecting proper relay with outdated channel state information. Numerical results demonstrate that our proposed Q-learning assisted relay selection scheme can achieve a significant improvement of effective secrecy throughput even with outdated channel information.