JEmpirical Dynamic Programming for Controlled Diffusion Processes

被引：0

作者：

Karumanchi, Sambhu H. ^{[1
]}

Belabbas, Mohamed A. ^{[2
]}

Hovakimyan, Naira ^{[3
]}

机构：

[1] Univ Illinois, Dept Aerosp Engn, Urbana, IL 61801 USA

[2] Univ Illinois, Dept Elect & Comp Engn, Urbana, IL USA

[3] Univ Illinois, Dept Mech Sci & Engn, Urbana, IL USA

来源：

IFAC PAPERSONLINE | 2023年 / 56卷 / 02期

基金：

美国国家科学基金会;

关键词：

Stochastic optimal control; Numerical methods for optimal control; Diffusion; processes; Reinforcement learning; Markov decision process; Value iteration; ALGORITHM;

D O I：

10.1016/j.ifacol.2023.10.854

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

We consider Markov chain approximation for optimal control of diffusion processes under infinite horizon discounted cost optimality and apply the simulation-based Empirical Value Iteration to estimate the value function of each approximating chain. We follow a nested multi-grid discretization of the state space to establish weak convergence of the value function sequence to the value function of the original controlled diffusion. We illustrate the convergence performance of the model on the popular Benes' bang-bang control problem [Bene.s (1974)].

引用

页码：11235 / 11241

页数：7

共 50 条

[1] ON THE DYNAMIC PROGRAMMING PRINCIPLE FOR CONTROLLED DIFFUSION PROCESSES IN A CYLINDRICAL REGION
Rokhlin, D. B.
SIBERIAN ELECTRONIC MATHEMATICAL REPORTS-SIBIRSKIE ELEKTRONNYE MATEMATICHESKIE IZVESTIYA, 2013, 10 : 302 - 310
[2] DYNAMIC PROGRAMMING PROCESSES WITHIN DYNAMIC PROGRAMMING PROCESSES
ROSE, CJ
JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS, 1969, 26 (03) : 669 - &
[3] A generalized dynamic programming procedure for controlled processes with profits
Ivaneshkin, Alexander I.
Journal of Automation and Information Sciences, 2007, 39 (03) : 61 - 67
[4] A DIRECT METHOD FOR STUDYING THE DYNAMIC-PROGRAMMING EQUATION FOR CONTROLLED DIFFUSION-PROCESSES IN HILBERT-SPACES
BARBU, V
DAPRATO, G
NUMERICAL FUNCTIONAL ANALYSIS AND OPTIMIZATION, 1981, 4 (01) : 23 - 43
[5] ROBUST ESTIMATION OF MULTIVARIATE JUMP-DIFFUSION PROCESSES VIA DYNAMIC PROGRAMMING
Torzhkov, Andrey
Sharma, Puneet
Chakraborty, Amit
PROCEEDINGS OF THE 2010 WINTER SIMULATION CONFERENCE, 2010, : 1123 - 1132
[6] Controlled diffusion processes
Borkar, Vivek S.
PROBABILITY SURVEYS, 2005, 2 : 213 - 244
[7] Linear Programming and the Control of Diffusion Processes
Ahn, Andrew
Haugh, Martin
INFORMS JOURNAL ON COMPUTING, 2015, 27 (04) : 646 - 657
[8] KINETICS OF PROCESSES CONTROLLED BY DIFFUSION
KAMINSKII, VA
BRUNOV, PA
TIMASHEV, SF
COLLOID JOURNAL OF THE USSR, 1977, 39 (04): : 608 - 613
[9] Approximate dynamic programming with Gaussian processes
Deisenroth, Marc P.
Peters, Jan
Rasmussen, Carl E.
2008 AMERICAN CONTROL CONFERENCE, VOLS 1-12, 2008, : 4480 - +
[10] Programming Elasticity and Commitment in Dynamic Processes
Fernandez, Pablo
Truong, Hong-Linh
Dustdar, Schahram
Ruiz-Cortes, Antonio
IEEE INTERNET COMPUTING, 2015, 19 (02) : 68 - 74

← 1 2 3 4 5 →