Asymptotic optimality of a class of controlled non-Markov processes

Motivated by applications in power systems and problems arising in simulation of large scale complex system optimizations, this work is concerned with controlled stochastic switching systems. The system of interest displays a multi-time scale structure. In contrast to the so-called singularly perturbed diffusions and multi-scale Markov decision processes, controlled non-Markov processes (also known as non-Markov decision processes) are treated. The novelty of our work is the treatment of the non-Markov controlled processes and the time-scale used. The fast and slow processes are coupled through a stochastic differential equation. Using averaging, it is first shown that the non-Markov switching process has a weak limit that is a Markov decision process. Then asymptotic optimal control of the non-Markov process is obtained by using the limit process.