SMDP-Based Prioritized Channel Allocations in Vehicular Ad Hoc Networks

In this paper, two semi-Markov decision process (SMDP)-based channel allocation methods are proposed for a vehicular ad hoc network (VANET) environment to maximize a long-term average system reward. First, we present a model-based dynamic programming method, which requires the knowledge of the system model, such as transition probabilities. After calculating the transition probabilities and time intervals, a relative value iteration algorithm is used to find the asymptotically optimal policy. Then, we propose a model-free reinforcement learning method in which we employ an agent to interact with the environment iteratively and learn from the feedback to approximate the optimal policy. Simulation results show that our reinforcement learning method can acquire a similar performance to that of the dynamic programming while both outperform the greedy method.