The complexity of decentralized control of Markov decision processes

被引：550

作者：

Bernstein, DS ^{[1
]}

Givan, R

Immerman, N

Zilberstein, S

机构：

[1] Univ Massachusetts, Dept Comp Sci, Amherst, MA 01003 USA

[2] Purdue Univ, Sch Elect & Comp Engn, W Lafayette, IN 47907 USA

来源：

MATHEMATICS OF OPERATIONS RESEARCH | 2002年 / 27卷 / 04期

关键词：

computational complexity; Markov decision process; decentralized control;

D O I：

10.1287/moor.27.4.819.297

中图分类号：

C93 [管理学]; O22 [运筹学];

学科分类号：

070105 ; 12 ; 1201 ; 1202 ; 120202 ;

摘要：

We consider decentralized control of Markov decision processes and give complexity bounds on the worst-case running time for algorithms that find optimal solutions. Generalizations of both the fully observable case and the partially observable case that allow for decentralized control are described. For even two agents, the finite-horizon problems corresponding to both of these models are hard for nondeterministic exponential time. These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov decision processes. In contrast to the problems involving centralized control, the problems we consider provably do not admit polynomial-time algorithms. Furthermore, assuming EXP not equal NEXP, the problems require superexponential time to solve in the worst case.

引用

页码：819 / 840

页数：22

共 50 条

[21] Complexity Bounds for Deterministic Partially Observed Markov Decision Processes
Vessaire, Cyrille
Carpentier, Pierre
Chancelier, Jean-Philippe
De Lara, Michel
Rodriguez-Martinez, Alejandro
ANNALS OF OPERATIONS RESEARCH, 2025, 344 (01) : 345 - 382
[22] Collaborative sensor management for multitarget tracking using decentralized Markov decision processes
Akselrod, D.
Goldman, C. V.
Sinha, A.
Kirubarajan, T.
SIGNAL AND DATA PROCESSING OF SMALL TARGETS 2006, 2006, 6236
[23] Actual Causality and Responsibility Attribution in Decentralized Partially Observable Markov Decision Processes
Triantafyllou, Stelios
Singla, Adish
Radanovic, Goran
PROCEEDINGS OF THE 2022 AAAI/ACM CONFERENCE ON AI, ETHICS, AND SOCIETY, AIES 2022, 2022, : 739 - 752
[24] Planning in Markov Decision Processes with Gap-Dependent Sample Complexity
Jonsson, Anders
Kaufmann, Emilie
Menard, Pierre
Domingues, Omar Darwiche
Leurent, Edouard
Valko, Michal
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020, 2020, 33
[25] Reducing Computational Complexity in Markov Decision Processes using Abstract Actions
Garcia-Hernandez, Ma. de Guadalupe
Ruiz-Pinales, Jose
Reyes-Ballesteros, Alberto
Onaindia, Eva
Avina-Cervantes, J. Gabriel
PROCEEDINGS OF THE SPECIAL SESSION OF THE SEVENTH MEXICAN INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE - MICAI 2008, 2008, : 256 - +
[26] The Complexity of Graph-Based Reductions for Reachability in Markov Decision Processes
Le Roux, Stephane
Perez, Guillermo A.
FOUNDATIONS OF SOFTWARE SCIENCE AND COMPUTATION STRUCTURES, FOSSACS 2018, 2018, 10803 : 367 - 383
[27] Monotone optimal control for a class of Markov decision processes
Zhuang, Weifen
Li, Michael Z. F.
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH, 2012, 217 (02) : 342 - 350
[28] Server Frequency Control Using Markov Decision Processes
Chen, Lydia Y.
Gautam, Natarajan
IEEE INFOCOM 2009 - IEEE CONFERENCE ON COMPUTER COMMUNICATIONS, VOLS 1-5, 2009, : 2951 - +
[29] Optimal control in light traffic Markov Decision Processes
INRIA, Sophia Antipolis, France
ZOR, 1 (63-79):
[30] Optimal control in light traffic Markov decision processes
Ger Koole
Olaf Passchier
Mathematical Methods of Operations Research, 1997, 45 : 63 - 79

← 1 2 3 4 5 →