Discrete-Time Local Value Iteration Adaptive Dynamic Programming: Admissibility and Termination Analysis

被引:32
|
作者
Wei, Qinglai [1 ]
Liu, Derong [2 ]
Lin, Qiao [1 ]
机构
[1] Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
[2] Univ Sci & Technol Beijing, Sch Automat & Elect Engn, Beijing 100083, Peoples R China
基金
中国国家自然科学基金;
关键词
Adaptive critic designs; adaptive dynamic programming (ADP); approximate dynamic programming; local iteration; neural networks; neurodynamic programming; nonlinear systems; optimal control; OPTIMAL TRACKING CONTROL; ZERO-SUM GAME; NONLINEAR-SYSTEMS; FEEDBACK-CONTROL; CONTROL SCHEME; LEARNING CONTROL; NETWORKS; DESIGN;
D O I
10.1109/TNNLS.2016.2593743
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this paper, a novel local value iteration adaptive dynamic programming (ADP) algorithm is developed to solve infinite horizon optimal control problems for discrete-time nonlinear systems. The focuses of this paper are to study admissibility properties and the termination criteria of discrete-time local value iteration ADP algorithms. In the discrete-time local value iteration ADP algorithm, the iterative value functions and the iterative control laws are both updated in a given subset of the state space in each iteration, instead of the whole state space. For the first time, admissibility properties of iterative control laws are analyzed for the local value iteration ADP algorithm. New termination criteria are established, which terminate the iterative local ADP algorithm with an admissible approximate optimal control law. Finally, simulation results are given to illustrate the performance of the developed algorithm.
引用
收藏
页码:2490 / 2502
页数:13
相关论文
共 50 条
  • [21] Convergence Analysis of Value Iteration Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems
    Xiao, Geyang
    Zhang, Huaguang
    [J]. IEEE TRANSACTIONS ON CYBERNETICS, 2024, 54 (03) : 1639 - 1649
  • [22] Policy Approximation in Policy Iteration Approximate Dynamic Programming for Discrete-Time Nonlinear Systems
    Guo, Wentao
    Si, Jennie
    Liu, Feng
    Mei, Shengwei
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2018, 29 (07) : 2794 - 2807
  • [23] A Generalized Policy Iteration Adaptive Dynamic Programming Algorithm for Optimal Control of Discrete-Time Nonlinear Systems with Actuator Saturation
    Lin, Qiao
    Wei, Qinglai
    Zhao, Bo
    [J]. ADVANCES IN NEURAL NETWORKS, PT II, 2017, 10262 : 60 - 65
  • [24] Optimal Learning Control for Discrete-Time Nonlinear Systems Using Generalized Policy Iteration Based Adaptive Dynamic Programming
    Wei, Qinglai
    Liu, Derong
    [J]. 2014 11TH WORLD CONGRESS ON INTELLIGENT CONTROL AND AUTOMATION (WCICA), 2014, : 1781 - 1786
  • [25] Discrete-Time ε-Adaptive Dynamic Programming Algorithm Using Neural Networks
    Jin, Ning
    Liu, Derong
    [J]. PROCEEDINGS OF THE 2008 IEEE INTERNATIONAL SYMPOSIUM ON INTELLIGENT CONTROL, 2008, : 114 - 119
  • [26] Adaptive Dynamic Programming for Discrete-Time Zero-Sum Games
    Wei, Qinglai
    Liu, Derong
    Lin, Qiao
    Song, Ruizhuo
    [J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2018, 29 (04) : 957 - 969
  • [27] A Novel Iterative θ-Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems
    Wei, Qinglai
    Liu, Derong
    [J]. IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2014, 11 (04) : 1176 - 1190
  • [28] Error Bound Analysis of Policy Iteration Based Approximate Dynamic Programming for Deterministic Discrete-time Nonlinear Systems
    Guo, Wentao
    Liu, Feng
    Si, Jennie
    Mei, Shengwei
    Li, Rui
    [J]. 2015 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2015,
  • [29] Generalized Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems: Convergence and Stability Analysis
    Liu, Derong
    Wei, Qinglai
    [J]. 2013 INTERNATIONAL CONFERENCE ON INFORMATION SCIENCE AND TECHNOLOGY (ICIST), 2013, : 134 - 141
  • [30] Hamiltonian-driven Adaptive Dynamic Programming for Nonlinear Discrete-Time Dynamic Systems
    Yang, Yongliang
    Wunsch, Donald
    Yin, Yixin
    [J]. 2017 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2017, : 1339 - 1346