Extreme point characterization of constrained nonstationary infinite-horizon Markov decision processes with finite state space

被引：1

作者：

Lee, Ilbin ^{[1
]}

Epelman, Marina A. ^{[1
]}

Romeijn, H. Edwin ^{[1
]}

Smith, Robert L. ^{[1
]}

机构：

[1] Univ Michigan, Ann Arbor, MI 48109 USA

来源：

OPERATIONS RESEARCH LETTERS | 2014年 / 42卷 / 03期

基金：

美国国家科学基金会;

关键词：

Extreme point; Markov decision process; Constrained optimization; Countably infinite linear program; POLICIES; SYSTEM;

D O I：

10.1016/j.orl.2014.03.001

中图分类号：

C93 [管理学]; O22 [运筹学];

学科分类号：

070105 ; 12 ; 1201 ; 1202 ; 120202 ;

摘要：

We study infinite-horizon nonstationary Markov decision processes with discounted cost criterion, finite state space, and side constraints. This problem can equivalently be formulated as a countably infinite linear program (CILP), a linear program with countably infinite number of variables and constraints. We provide a complete algebraic characterization of extreme points of the CILP formulation and illustrate the characterization for special cases. The existence of a K-randomized optimal policy for a problem with K side constraints also follows from this characterization. (C) 2014 Elsevier B.V. All rights reserved.

引用

页码：238 / 245

页数：8

共 50 条

[1] A Linear Programming Approach to Nonstationary Infinite-Horizon Markov Decision Processes
Ghate, Archis
Smith, Robert L.
[J]. OPERATIONS RESEARCH, 2013, 61 (02) : 413 - 425
[2] FINITE-STATE APPROXIMATIONS FOR DENUMERABLE-STATE INFINITE-HORIZON DISCOUNTED MARKOV DECISION-PROCESSES
WHITE, DJ
[J]. JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 1980, 74 (01) : 292 - 295
[3] DUAL ASCENT AND PRIMAL-DUAL ALGORITHMS FOR INFINITE-HORIZON NONSTATIONARY MARKOV DECISION PROCESSES
Ghate, Archis
[J]. SIAM JOURNAL ON OPTIMIZATION, 2023, 33 (03) : 1391 - 1415
[4] Model-Based Reinforcement Learning for Infinite-Horizon Discounted Constrained Markov Decision Processes
HasanzadeZonuzy, Aria
Kalathil, Dileep
Shakkottai, Srinivas
[J]. PROCEEDINGS OF THE THIRTIETH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2021, 2021, : 2519 - 2525
[5] Characterization and computation of infinite-horizon specifications over Markov processes
Tkachev, Ilya
Abate, Alessandro
[J]. THEORETICAL COMPUTER SCIENCE, 2014, 515 : 1 - 18
[6] Cosine Policy Iteration for Solving Infinite-Horizon Markov Decision Processes
Frausto-Solis, Juan
Santiago, Elizabeth
Mora-Vargas, Jaime
[J]. MICAI 2009: ADVANCES IN ARTIFICIAL INTELLIGENCE, PROCEEDINGS, 2009, 5845 : 75 - +
[7] Fast Approximate Dynamic Programming for Infinite-Horizon Markov Decision Processes
Kolarijani, M. A. S.
Max, G. F.
Esfahani, P. Mohajerin
[J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 34 (NEURIPS 2021), 2021, 34
[8] Solution and forecast horizons for infinite-horizon nonhornogeneous Markov decision processes
Cheevaprawatdomrong, Torpong
Schochetman, Irwin E.
Smith, Robert L.
Garcia, Alfredo
[J]. MATHEMATICS OF OPERATIONS RESEARCH, 2007, 32 (01) : 51 - 72
[9] FINITE STATE CONTINUOUS TIME MARKOV DECISION PROCESSES WITH AN INFINITE PLANNING HORIZON
MILLER, BL
[J]. JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 1968, 22 (03) : 552 - &
[10] Learning Infinite-Horizon Average-Reward Markov Decision Processes with Constraints
Chen, Liyu
Jain, Rahul
Luo, Haipeng
[J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162, 2022,

← 1 2 3 4 5 →