Reinforcement-Learning-Based Double Auction Design for Dynamic Spectrum Access in Cognitive Radio Networks

In cognitive radio networks, an important issue is to share the detected available spectrum among different secondary users to improve the network performance. Although some work has been done for dynamic spectrum access, the learning capability of cognitive radio networks is largely ignored in the previous work. In this paper, we propose a reinforcement-learning-based double auction algorithm aiming to improve the performance of dynamic spectrum access in cognitive radio networks. The dynamic spectrum access process is modeled as a double auction game. Based on the spectrum access history information, both primary users and secondary users can estimate the impact on their future rewards and then adapt their spectrum access or release strategies effectively to compete for channel opportunities. Simulation results show that the proposed reinforcement-learning-based double auction algorithm can significantly improve secondary users' performance in terms of packet loss, bidding efficiency and transmission rate or opportunity access.