Performance optimization of semi-Markov decision processes with discounted-cost criteria

被引：3

作者：

Yin, Baoqun ^{[1
]}

Li, Yanjie ^{[1
]}

Zhou, Yaping ^{[1
]}

Xi, Hongsheng ^{[1
]}

机构：

[1] Univ Sci & Technol China, Dept Automat, Hefei 230026, Anhui, Peoples R China

来源：

EUROPEAN JOURNAL OF CONTROL | 2008年 / 14卷 / 03期

关键词：

semi-Markov decision processes; discounted Poisson equation; alpha-potential; discounted-cost criteria; policy iteration; value iteration;

D O I：

10.3166/EJC.14.213-222

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

We discuss the problems of discounted-cost performance optimization for a class of semi-Markov decision processes (SMDPs). We define a matrix which can be used as the infinitesimal generator of a Markov process. The discounted Poisson equation is proposed for an SMDP by using this matrix, from which the alpha-potential is defined. The optimally equation satisfied by the optimal stationary policy is given and the relation between discounted model and average model is discussed. Two iteration algorithms to find is an element of-optimal policies are proposed and the proofs of convergence of these two algorithms are given. A numerical example is provided to illustrate the application of the algorithms.

引用

页码：213 / 222

页数：10

共 50 条

[21] A UNIFIED APPROACH TO ALGORITHMS WITH A SUBOPTIMALITY TEST IN DISCOUNTED SEMI-MARKOV DECISION-PROCESSES
OHNO, K
[J]. JOURNAL OF THE OPERATIONS RESEARCH SOCIETY OF JAPAN, 1981, 24 (04) : 296 - 324
[22] First passage models for denumerable semi-Markov decision processes with nonnegative discounted costs
Huang, Yong-hui
Guo, Xian-ping
[J]. ACTA MATHEMATICAE APPLICATAE SINICA-ENGLISH SERIES, 2011, 27 (02): : 177 - 190
[23] Optimality of Quasi-Open-Loop Policies for Discounted Semi-Markov Decision Processes
Adelman, Daniel
Mancini, Angelo J.
[J]. MATHEMATICS OF OPERATIONS RESEARCH, 2016, 41 (04) : 1222 - 1247
[24] Error bounds of optimization algorithms for semi-Markov decision processes
Tang, Hao
Yin, Baoqun
Xi, Hongsheng
[J]. INTERNATIONAL JOURNAL OF SYSTEMS SCIENCE, 2007, 38 (09) : 725 - 736
[25] Hierarchical Dialogue Optimization Using Semi-Markov Decision Processes
Cuayahuitl, Heriberto
Renals, Steve
Lemon, Oliver
Shimodaira, Hiroshi
[J]. INTERSPEECH 2007: 8TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION, VOLS 1-4, 2007, : 1413 - +
[26] NONSTATIONARY CONTINUOUS-TIME MARKOV DECISION-PROCESSES IN A SEMI-MARKOV ENVIRONMENT WITH DISCOUNTED CRITERION
HU, QY
[J]. JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 1995, 194 (03) : 640 - 659
[27] COMPUTATION OF OPTIMAL POLICIES IN DISCOUNTED SEMI-MARKOV DECISION CHAINS
CANTALUPPI, L
[J]. OR SPEKTRUM, 1984, 6 (03) : 147 - 160
[28] Discounted cost Markov decision processes with a constraint
Wakuta, K
[J]. PROBABILITY IN THE ENGINEERING AND INFORMATIONAL SCIENCES, 1998, 12 (02) : 177 - 187
[29] THE EXPONENTIAL COST OPTIMALITY FOR FINITE HORIZON SEMI-MARKOV DECISION PROCESSES
Huo, Haifeng
Wen, Xian
[J]. KYBERNETIKA, 2022, 58 (03) : 301 - 319
[30] Hierarchical optimization of policy-coupled semi-Markov decision processes
Wang, G
Mahadevan, S
[J]. MACHINE LEARNING, PROCEEDINGS, 1999, : 464 - 473

← 1 2 3 4 5 →