A neuro-dynamic programming approach to admission control in ATM networks: The single link case

We are interested in solving large-scale Markov Decision Problems. The classical method of Dynamic Programming provides a mathematical framework for finding optimal solutions for a given Markov Decision Problem. However, for Dynamic Programming algorithms become computationally infeasible when the underlying Markov Decision Problem evolves over a large state space. In recent years, a new methodology, called Neuro-Dynamic Programming, has emerged which tries to overcome this ''curse of dimensionality''. We present how Neuro-Dynamic Programming can be applied to the Admission Control Problem for a single link in an ATM environment. Based on results obtained through Neuro-Dynamic Programming, we derive a heuristic ''Threshold'' policy. Performances of the policies obtained through Neuro-Dynamic Programming are compared with a policy which always accepts a customer when the required resources are available.