A Version of the Euler Equation in Discounted Markov Decision Processes

被引：2

作者：

Cruz-Suarez, H. ^{[1
]}

Zacarias-Espinoza, G. ^{[1
]}

Vazquez-Guevara, V. ^{[1
]}

机构：

[1] Benemerita Univ Autonoma Puebla, Fac Ciencias Fis Matemat, CU, Puebla 72570, PUE, Mexico

来源：

JOURNAL OF APPLIED MATHEMATICS | 2012年

关键词：

UNCERTAINTY; GROWTH;

D O I：

10.1155/2012/103698

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

This paper deals with Markov decision processes (MDPs) on Euclidean spaces with an infinite horizon. An approach to study this kind of MDPs is using the dynamic programming technique (DP). Then the optimal value function is characterized through the value iteration functions. The paper provides conditions that guarantee the convergence of maximizers of the value iteration functions to the optimal policy. Then, using the Euler equation and an envelope formula, the optimal solution of the optimal control problem is obtained. Finally, this theory is applied to a linear-quadratic control problem in order to find its optimal policy.

引用

页数：16

共 50 条

[11] A note on deterministic approximation of discounted Markov decision processes
Cruz-Suarez, Hugo
Gordienko, Evgueni
Montes-de-Oca, Raul
APPLIED MATHEMATICS LETTERS, 2009, 22 (08) : 1252 - 1256
[12] Constrained discounted Markov decision processes and Hamiltonian Cycles
Feinberg, EA
MATHEMATICS OF OPERATIONS RESEARCH, 2000, 25 (01) : 130 - 140
[13] Constrained discounted Markov decision processes and Hamiltonian cycles
Feinberg, EA
PROCEEDINGS OF THE 36TH IEEE CONFERENCE ON DECISION AND CONTROL, VOLS 1-5, 1997, : 2821 - 2826
[14] Discounted Markov Decision Processes via Time Aggregation
Arruda, Edilson F.
Fragoso, Marcelo D.
2016 EUROPEAN CONTROL CONFERENCE (ECC), 2016, : 2578 - 2583
[15] Stochastic approximations of constrained discounted Markov decision processes
Dufour, Francois
Prieto-Rumeau, Tomas
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 2014, 413 (02) : 856 - 879
[16] Hierarchical algorithms for discounted and weighted Markov decision processes
M. Abbad
C. Daoui
Mathematical Methods of Operations Research, 2003, 58 : 237 - 245
[17] Limits of multi-discounted Markov decision processes
Gimbert, Hugo
Zielonka, Wieslaw
22ND ANNUAL IEEE SYMPOSIUM ON LOGIC IN COMPUTER SCIENCE, PROCEEDINGS, 2007, : 89 - +
[18] Decision roll and horizon roll processes in infinite horizon discounted Markov decision processes
White, DJ
MANAGEMENT SCIENCE, 1996, 42 (01) : 37 - 50
[19] The Discounted Euler Equation: A Note
McKay, Alisdair
Nakamura, Emi
Steinsson, Jon
ECONOMICA, 2017, 84 (336) : 820 - 831
[20] Discounted Deterministic Markov Decision Processes and Discounted All-Pairs Shortest Paths
Madani, Omid
Thorup, Mikkel
Zwick, Uri
ACM TRANSACTIONS ON ALGORITHMS, 2010, 6 (02)

← 1 2 3 4 5 →