Optimization in distributed controlled Markov chains

The performance potential theory has proved to be a promising tool in optimizing the infinite-horizon Markov decision problem (MDP) [1]-[2] So far, the research in this area is implicitly focused on a simple system with a single controller In this paper, we consider the distributed controlled Markov chain, where the system consists of several individual control units and it evolves under the combined control of these nodes. Motivated by practical background, we investigate a structure of MDP with event-dependent decisions. We explore a notion of expanded Markov chain to map this problem to a traditional MDP model. In particular, we address ourselves to the complexity-reduction techniques to deal with the enlarged state space. For the distributed system where a particular node can only access partial system information, we develop some algorithms for decentralized potential estimation and policy iteration.