Efficient model-based reinforcement learning for approximate online optimal control

被引：61
|
作者：
Kamalapurkar, Rushikesh [1 ]
Rosenfeld, Joel A. [2 ]
Dixon, Warren E. [2 ]
机构：
[1] Oklahoma State Univ, Sch Mech & Aerosp Engn, Stillwater, OK 74078 USA
[2] Univ Florida, Dept Mech & Aerosp Engn, Gainesville, FL USA
来源：
AUTOMATICA | 2016年 / 74卷
基金：
美国国家科学基金会;
关键词：
Model-based reinforcement learning; Data-based control; Adaptive control; Local approximation; DISCRETE-TIME-SYSTEMS; ADAPTIVE OPTIMAL-CONTROL; NONLINEAR-SYSTEMS; NETWORK; DYNAMICS;
D O I：
10.1016/j.automatica.2016.08.004
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
An infinite horizon optimal regulation problem is solved online for a deterministic control-affine nonlinear dynamical system using a state following (StaF) kernel method to approximate the value function. Unlike traditional methods that aim to approximate a function over a large compact set, the StaF kernel method aims to approximate a function in a small neighborhood of a state that travels within a compact set. Simulation results demonstrate that stability and approximate optimality of the control system can be achieved with significantly fewer basis functions than may be required for global approximation methods. (C) 2016 Elsevier Ltd. All rights reserved.
引用
收藏
页码：247 / 258
页数：12
相关论文
共 50 条
  • [1] Model-based reinforcement learning for approximate optimal regulation
    Kamalapurkar, Rushikesh
    Walters, Patrick
    Dixon, Warren E.
    [J]. AUTOMATICA, 2016, 64 : 94 - 104
  • [2] Model-Based Reinforcement Learning for Approximate Optimal Control with Temporal Logic Specifications
    Cohen, Max H.
    Belta, Cahn
    [J]. HSCC2021: PROCEEDINGS OF THE 24TH INTERNATIONAL CONFERENCE ON HYBRID SYSTEMS: COMPUTATION AND CONTROL (PART OF CPS-IOT WEEK), 2021,
  • [3] Cooperative Model-Based Reinforcement Learning for Approximate Optimal Tracking
    Greene, Max L.
    Bell, Zachary, I
    Nivison, Scott A.
    How, Jonathan P.
    Dixon, Warren E.
    [J]. 2021 AMERICAN CONTROL CONFERENCE (ACC), 2021, : 1973 - 1978
  • [4] Efficient reinforcement learning: Model-based acrobot control
    Boone, G
    [J]. 1997 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION - PROCEEDINGS, VOLS 1-4, 1997, : 229 - 234
  • [5] Model-Based Reinforcement Learning for Infinite-Horizon Approximate Optimal Tracking
    Kamalapurkar, Rushikesh
    Andrews, Lindsey
    Walters, Patrick
    Dixon, Warren E.
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2017, 28 (03) : 753 - 758
  • [6] Model-based reinforcement learning for infinite-horizon approximate optimal tracking
    Kamalapurkar, Rushikesh
    Andrews, Lindsey
    Walters, Patrick
    Dixon, Warren E.
    [J]. 2014 IEEE 53RD ANNUAL CONFERENCE ON DECISION AND CONTROL (CDC), 2014, : 5083 - 5088
  • [7] Online Constrained Model-based Reinforcement Learning
    van Niekerk, Benjamin
    Damianou, Andreas
    Rosman, Benjamin
    [J]. CONFERENCE ON UNCERTAINTY IN ARTIFICIAL INTELLIGENCE (UAI2017), 2017,
  • [8] Model-Based Reinforcement Learning for Optimal Feedback Control of Switched Systems
    Greene, Max L.
    Abudia, Moad
    Kamalapurkar, Rushikesh
    Dixon, Warren E.
    [J]. 2020 59TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC), 2020, : 162 - 167
  • [9] Model-Based Reinforcement Learning for Time-Optimal Velocity Control
    Hartmann, Gabriel
    Shiller, Zvi
    Azaria, Amos
    [J]. IEEE ROBOTICS AND AUTOMATION LETTERS, 2020, 5 (04): : 6185 - 6192
  • [10] Sample-efficient model-based reinforcement learning for quantum control
    Khalid, Irtaza
    Weidner, Carrie A.
    Jonckheere, Edmond A.
    Schirmer, Sophie G.
    Langbein, Frank C.
    [J]. PHYSICAL REVIEW RESEARCH, 2023, 5 (04):
12345