Delay-Optimal Distributed Resource Allocation for Device-to-Device Communications

In this paper, the resource allocation problem is investigated for Device-to-Device (D2D) communications underlaying cellular networks with bursty traffic arrival. We formulate an infinite horizon average reward constraint Markov decision process (CMDP) that aims at minimizing the average delay under the dropping propability constraint. Then, we present a reduced-state Bellman's equation with linear value function approximation to deal with the curse of dimensionality problem in solving the CMDP. A distributed resource allocation algorithm is derived with low computation complexity and signaling overhead, which consists of a subchannel bidding mechanism to obtain the optimal control action, and a distributed online stochastic learning algorithm to estimate the value function and the optimal Lagrangian Multipliers (LMs). Simulation results show that the performance of our proposed algorithm is very close to that achieved by the offline value iteration algorithm, and is better than various baselines algorithms.