Reinforcement learning-based spectrum handoff scheme with measured PDR in cognitive radio networks

Spectrum handoff plays an important role in cognitive radio networks (CRNs). Secondary users (SUs) use spectrum handoff to hold on the idle channel or to free the channel for primary users (PUs). Spectrum handoff scheme greatly affects the transmission quality and the success rate of SUs connection. In this Letter, a reinforcement learning-based spectrum handoff scheme with the measured packet drop rate (PDR) for multimedia transmissions over CRNs is proposed. In a system model with multiple PUs and SUs, a new state space description method is designed and an observed state includes not only the status whether PUs arrive on each channel but also several other important factors. Also, the measured PDR, instead of the calculated one, is presented to update the mean opinion score, the Q-table and the handoff policy. Compared with the existing schemes with the calculated PDR from the Quality-of-Experience model, the authors' proposed scheme can converge more rapidly in the dynamic radio environment, and reduce the PDR of SUs more significantly.