Relaxation for Constrained Decentralized Markov Decision Processes

This paper studies a class of decentralized multi-agent stochastic optimization problems. In these problems, each agent has only a partial view of the world state, and a partial control of the actions but must cooperatively maximize the long-term system reward. The state that an agent observe consists of two parts-a common public component and an agent-specific private component. Importantly, taking actions incurs costs and the actions that the agents can take are subject to an overall cost constraint in each interaction period. We formulate this problem as an infinite time horizon Decentralized Markov Decision Process (DEC-MDP) with resource constraints and develop efficient approximate algorithms that allow decentralized computation of the agent policy based on Lagrangian relaxation.