Extreme point characterization of constrained nonstationary infinite-horizon Markov decision processes with finite state space

被引：1

作者：

Lee, Ilbin ^{[1
]}

Epelman, Marina A. ^{[1
]}

Romeijn, H. Edwin ^{[1
]}

Smith, Robert L. ^{[1
]}

机构：

[1] Univ Michigan, Ann Arbor, MI 48109 USA

来源：

OPERATIONS RESEARCH LETTERS | 2014年 / 42卷 / 03期

基金：

美国国家科学基金会;

关键词：

Extreme point; Markov decision process; Constrained optimization; Countably infinite linear program; POLICIES; SYSTEM;

D O I：

10.1016/j.orl.2014.03.001

中图分类号：

C93 [管理学]; O22 [运筹学];

学科分类号：

070105 ; 12 ; 1201 ; 1202 ; 120202 ;

摘要：

We study infinite-horizon nonstationary Markov decision processes with discounted cost criterion, finite state space, and side constraints. This problem can equivalently be formulated as a countably infinite linear program (CILP), a linear program with countably infinite number of variables and constraints. We provide a complete algebraic characterization of extreme points of the CILP formulation and illustrate the characterization for special cases. The existence of a K-randomized optimal policy for a problem with K side constraints also follows from this characterization. (C) 2014 Elsevier B.V. All rights reserved.

引用

页码：238 / 245

页数：8

共 50 条

[31] FINITE STATE CONTINUOUS TIME MARKOV DECISION PROCESSES WITH A FINITE PLANNING HORIZON
MILLER, BL
[J]. SIAM JOURNAL ON CONTROL, 1968, 6 (02): : 266 - &
[32] Solving an Infinite-Horizon Discounted Markov Decision Process by DC Programming and DCA
Vinh Thanh Ho
Hoai An Le Thi
[J]. ADVANCED COMPUTATIONAL METHODS FOR KNOWLEDGE ENGINEERING (ICCSAMA 2016), 2016, 453 : 43 - 55
[33] Infinite-Horizon Average-Cost Markov Decision Process Routing Games
Calderone, Dan
Sastry, S. Shankar
[J]. 2017 IEEE 20TH INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC), 2017,
[34] A Remark on Finite Horizon Markov Decision processes
XikUi Wang (University of Saskatchewan
[J]. 经济数学, 1989, (05) : 76 - 80
[35] On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems
Madani, O
Hanks, S
Condon, A
[J]. SIXTEENTH NATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE (AAAI-99)/ELEVENTH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE (IAAI-99), 1999, : 541 - 548
[36] Policy-based branch-and-bound for infinite-horizon Multi-model Markov decision processes
Ahluwalia, Vinayak S.
Steimle, Lauren N.
Denton, Brian T.
[J]. COMPUTERS & OPERATIONS RESEARCH, 2021, 126 (126)
[37] Constrained optimality for finite horizon semi-Markov decision processes in Polish spaces
Huang, Yonghui
Li, Zhongfei
Guo, Xianping
[J]. OPERATIONS RESEARCH LETTERS, 2014, 42 (02) : 123 - 129
[38] BACKWARD REPRESENTATION FOR NONSTATIONARY MARKOV-PROCESSES WITH FINITE-STATE SPACE
DIMASI, GB
KITSUL, PI
[J]. SYSTEMS & CONTROL LETTERS, 1994, 22 (06) : 445 - 450
[39] Approximate robust policy iteration for discounted infinite-horizon Markov decision processes with uncertain stationary parametric tiransition matrices
Li, Baohua
Si, Jennie
[J]. 2007 IEEE INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOLS 1-6, 2007, : 2052 - 2057
[40] Belief Function Model for Reliable Optimal Set Estimation of Transition Matrices in Discounted Infinite-Horizon Markov Decision Processes
Li, Baohua
Si, Jennie
[J]. 2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2011, : 1214 - 1221

← 1 2 3 4 5 →