Markov Decision Processes with Average-Value-at-Risk criteria

被引：79

作者：

Baeuerle, Nicole ^{[1
]}

Ott, Jonathan ^{[1
]}

机构：

[1] Karlsruhe Inst Technol, Inst Stochast, D-76128 Karlsruhe, Germany

来源：

MATHEMATICAL METHODS OF OPERATIONS RESEARCH | 2011年 / 74卷 / 03期

关键词：

Markov Decision Problem; Average-Value-at-Risk; Time-consistency; Risk aversion; TIME; OPTIMIZATION; VARIANCE;

D O I：

10.1007/s00186-011-0367-0

中图分类号：

C93 [管理学]; O22 [运筹学];

学科分类号：

070105 ; 12 ; 1201 ; 1202 ; 120202 ;

摘要：

We investigate the problem of minimizing the Average-Value-at-Risk (AVaR(tau)) of the discounted cost over a finite and an infinite horizon which is generated by a Markov Decision Process (MDP). We show that this problem can be reduced to an ordinary MDP with extended state space and give conditions under which an optimal policy exists. We also give a time-consistent interpretation of the AVaR(tau). At the end we consider a numerical example which is a simple repeated casino game. It is used to discuss the influence of the risk aversion parameter tau of the AVaR(tau)-criterion.

引用

页码：361 / 379

页数：19

共 50 条

[1] Markov Decision Processes with Average-Value-at-Risk criteria
Nicole Bäuerle
Jonathan Ott
Mathematical Methods of Operations Research, 2011, 74 : 361 - 379
[2] An average-value-at-risk criterion for Markov decision processes with unbounded costs
Liu, Qiuli
Ching, Wai-Ki
Zhang, Junyu
Wang, Hongchu
FRONTIERS OF MATHEMATICS IN CHINA, 2022, 17 (04) : 673 - 687
[3] An average-value-at-risk criterion for Markov decision processes with unbounded costs
Qiuli Liu
Wai-Ki Ching
Junyu Zhang
Hongchu Wang
Frontiers of Mathematics in China, 2022, 17 : 673 - 687
[4] Risk Aversion in Finite Markov Decision Processes Using Total Cost Criteria and Average Value at Risk
Carpin, Stefano
Chow, Yin-Lam
Pavone, Marco
2016 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2016, : 335 - 342
[5] Limiting average criteria for nonstationary Markov decision processes
Guo, XP
Shi, P
SIAM JOURNAL ON OPTIMIZATION, 2001, 11 (04) : 1037 - 1053
[6] A new coherent multivariate average-value-at-risk
Ugurlu, Kerem
OPTIMIZATION, 2023, 72 (02) : 493 - 519
[7] Constrained continuous-time Markov decision processes with average criteria
Lanlan Zhang
Xianping Guo
Mathematical Methods of Operations Research, 2008, 67 : 323 - 340
[8] PERCENTILE PERFORMANCE CRITERIA FOR LIMITING AVERAGE MARKOV DECISION-PROCESSES
FILAR, JA
KRASS, D
ROSS, KW
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 1995, 40 (01) : 2 - 10
[9] Constrained continuous-time Markov decision processes with average criteria
Zhang, Lanlan
Guo, Xianping
MATHEMATICAL METHODS OF OPERATIONS RESEARCH, 2008, 67 (02) : 323 - 340
[10] The Discounted Method and Equivalence of Average Criteria for Risk-Sensitive Markov Decision Processes on Borel Spaces
Rolando Cavazos-Cadena
Francisco Salem-Silva
Applied Mathematics and Optimization, 2010, 61 : 167 - 190

← 1 2 3 4 5 →