Markov Decision Processes with Average-Value-at-Risk criteria

被引：79

作者：

Baeuerle, Nicole ^{[1
]}

Ott, Jonathan ^{[1
]}

机构：

[1] Karlsruhe Inst Technol, Inst Stochast, D-76128 Karlsruhe, Germany

来源：

MATHEMATICAL METHODS OF OPERATIONS RESEARCH | 2011年 / 74卷 / 03期

关键词：

Markov Decision Problem; Average-Value-at-Risk; Time-consistency; Risk aversion; TIME; OPTIMIZATION; VARIANCE;

D O I：

10.1007/s00186-011-0367-0

中图分类号：

C93 [管理学]; O22 [运筹学];

学科分类号：

070105 ; 12 ; 1201 ; 1202 ; 120202 ;

摘要：

We investigate the problem of minimizing the Average-Value-at-Risk (AVaR(tau)) of the discounted cost over a finite and an infinite horizon which is generated by a Markov Decision Process (MDP). We show that this problem can be reduced to an ordinary MDP with extended state space and give conditions under which an optimal policy exists. We also give a time-consistent interpretation of the AVaR(tau). At the end we consider a numerical example which is a simple repeated casino game. It is used to discuss the influence of the risk aversion parameter tau of the AVaR(tau)-criterion.

引用

页码：361 / 379

页数：19

共 50 条

[21] A pause control approach to the value iteration scheme in average Markov decision processes
Cavazos-Cadena, Rolando
Systems and Control Letters, 1998, 33 (04): : 209 - 219
[22] A pause control approach to the value iteration scheme in average Markov decision processes
Cavazos-Cadena, R
SYSTEMS & CONTROL LETTERS, 1998, 33 (04) : 209 - 219
[23] Toward an Optimized Value Iteration Algorithm for Average Cost Markov Decision Processes
Arruda, Edilson F.
Ourique, Fabricio
Almudevar, Anthony
49TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC), 2010, : 930 - 934
[24] Value Iteration for Long-Run Average Reward in Markov Decision Processes
Ashok, Pranav
Chatterjee, Krishnendu
Daca, Przemyslaw
Kretinsky, Jan
Meggendorfer, Tobias
COMPUTER AIDED VERIFICATION, CAV 2017, PT I, 2017, 10426 : 201 - 221
[25] Unbounded cost Markov decision processes with limsup and liminf average criteria: new conditions
Quanxin Zhu
Xianping Guo
Yonglong Dai
Mathematical Methods of Operations Research, 2005, 61 : 469 - 482
[26] Unbounded cost Markov decision processes with limsup and liminf average criteria:: new conditions
Zhu, QX
Guo, XP
Dai, YL
MATHEMATICAL METHODS OF OPERATIONS RESEARCH, 2005, 61 (03) : 469 - 482
[27] A Unified Approach for Semi-Markov Decision Processes with Discounted and Average Reward Criteria
Li, Yanjie
Wang, Huijing
Chen, Haoyao
2014 11TH WORLD CONGRESS ON INTELLIGENT CONTROL AND AUTOMATION (WCICA), 2014, : 1741 - 1744
[28] MINIMUM AVERAGE VALUE-AT-RISK FOR FINITE HORIZON SEMI-MARKOV DECISION PROCESSES IN CONTINUOUS TIME
Huang, Yonghui
Guo, Xianping
SIAM JOURNAL ON OPTIMIZATION, 2016, 26 (01) : 1 - 28
[29] Credibilistic Markov decision processes: The average case
Kageyama, Masayuki
JOURNAL OF COMPUTATIONAL AND APPLIED MATHEMATICS, 2009, 224 (01) : 140 - 145
[30] VALUE ITERATION IN COUNTABLE STATE AVERAGE COST MARKOV DECISION PROCESSES WITH UNBOUNDED COSTS
Sennott, Linn I.
ANNALS OF OPERATIONS RESEARCH, 1991, 28 (01) : 261 - 271

← 1 2 3 4 5 →