A note on optimality conditions for continuous-time Markov decision processes with average cost criterion

被引:25
|
作者
Guo, XP [1 ]
Liu, K
机构
[1] Zhongshan Univ, Dept Math, Guangzhou, Peoples R China
[2] Asia Pacific Operat Res Ctr, Seoul, South Korea
[3] Chinese Acad Sci, Acad Math & Syst Sci, Inst Appl Math, Beijing 100864, Peoples R China
基金
中国国家自然科学基金;
关键词
average cost criterion; continuous-time Markov decision processes (MDPs); optimal stationary policies; optimality inequality;
D O I
10.1109/9.975505
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This note deals with continuous-time Markov decision processes with a denumerable state space and the average cost criterion. The transition rates are allowed to be unbounded, and the action set is a Borel space. We give a new set of conditions under which the existence of optimal stationary policies is ensured by using the optimality inequality. Our results are illustrated with a controlled queueing model. Moreover, we use an example to show that our conditions do not imply the existence of a solution to the optimality equations in the previous literature.
引用
收藏
页码:1984 / 1989
页数:6
相关论文
共 50 条
  • [21] Bias optimality for multichain continuous-time Markov decision processes
    Guo, Xianping
    Song, XinYuan
    Zhang, Junyu
    [J]. OPERATIONS RESEARCH LETTERS, 2009, 37 (05) : 317 - 321
  • [22] Constrained continuous-time Markov decision processes with average criteria
    Lanlan Zhang
    Xianping Guo
    [J]. Mathematical Methods of Operations Research, 2008, 67 : 323 - 340
  • [23] Constrained continuous-time Markov decision processes with average criteria
    Zhang, Lanlan
    Guo, Xianping
    [J]. MATHEMATICAL METHODS OF OPERATIONS RESEARCH, 2008, 67 (02) : 323 - 340
  • [24] Discounted optimality for continuous-time Markov decision processes in Polish spaces
    Guo, Xianping
    [J]. 2006 CHINESE CONTROL CONFERENCE, VOLS 1-5, 2006, : 1785 - 1787
  • [25] Mean-semivariance optimality for continuous-time Markov decision processes
    Wei, Qingda
    [J]. SYSTEMS & CONTROL LETTERS, 2019, 125 : 67 - 74
  • [26] The risk probability criterion for discounted continuous-time Markov decision processes
    Huo, Haifeng
    Zou, Xiaolong
    Guo, Xianping
    [J]. DISCRETE EVENT DYNAMIC SYSTEMS-THEORY AND APPLICATIONS, 2017, 27 (04): : 675 - 699
  • [27] The risk probability criterion for discounted continuous-time Markov decision processes
    Haifeng Huo
    Xiaolong Zou
    Xianping Guo
    [J]. Discrete Event Dynamic Systems, 2017, 27 : 675 - 699
  • [28] NONSTATIONARY CONTINUOUS-TIME MARKOV DECISION-PROCESSES WITH DISCOUNTED CRITERION
    HU, QY
    [J]. JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 1993, 180 (01) : 60 - 70
  • [29] Denumerable continuous-time Markov decision processes with multiconstraints on average costs
    Liu, Qiuli
    Tan, Hangsheng
    Guo, Xianping
    [J]. INTERNATIONAL JOURNAL OF SYSTEMS SCIENCE, 2012, 43 (03) : 576 - 585
  • [30] Continuous-time Markov decision processes with nth-bias optimality criteria
    Zhang, Junyu
    Cao, Xi-Ren
    [J]. AUTOMATICA, 2009, 45 (07) : 1628 - 1638