Continuous-Time Distributed Policy Iteration for Multicontroller Nonlinear Systems

被引：76
|
作者：
Wei, Qinglai [1 ,2 ,3 ]
Li, Hongyang [1 ,2 ,3 ]
Yang, Xiong [4 ]
He, Haibo [5 ]
机构：
[1] Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
[2] Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
[3] Qingdao Acad Intelligent Ind, Qingdao 266109, Peoples R China
[4] Tianjin Univ, Sch Elect & Informat Engn, Tianjin 300072, Peoples R China
[5] Univ Rhode Isl, Dept Elect Comp & Biomed Engn, Kingston, RI 02881 USA
来源：
IEEE TRANSACTIONS ON CYBERNETICS | 2021年 / 51卷 / 05期
基金：
中国国家自然科学基金; 美国国家科学基金会;
关键词：
Optimal control; Nonlinear systems; Decentralized control; Mathematical model; Convergence; Multi-agent systems; Adaptive dynamic programming (ADP); approximate dynamic programming; distributed policy iteration; nonlinear systems; optimal control; ZERO-SUM GAMES; MULTIAGENT SYSTEMS; TRACKING CONTROL; DRIVEN;
D O I：
10.1109/TCYB.2020.2979614
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
In this article, a novel distributed policy iteration algorithm is established for infinite horizon optimal control problems of continuous-time nonlinear systems. In each iteration of the developed distributed policy iteration algorithm, only one controller's control law is updated and the other controllers' control laws remain unchanged. The main contribution of the present algorithm is to improve the iterative control law one by one, instead of updating all the control laws in each iteration of the traditional policy iteration algorithms, which effectively releases the computational burden in each iteration. The properties of distributed policy iteration algorithm for continuous-time nonlinear systems are analyzed. The admissibility of the present methods has also been analyzed. Monotonicity, convergence, and optimality have been discussed, which show that the iterative value function is nonincreasingly convergent to the solution of the Hamilton-Jacobi-Bellman equation. Finally, numerical simulations are conducted to illustrate the effectiveness of the proposed method.
引用
收藏
页码：2372 / 2383
页数：12
相关论文
共 50 条
  • [1] On approximate policy iteration for continuous-time systems
    Wernrud, Andreas
    Rantzer, Anders
    [J]. 2005 44th IEEE Conference on Decision and Control & European Control Conference, Vols 1-8, 2005, : 1453 - 1458
  • [2] Generalized Policy Iteration for Continuous-Time Systems
    Vrabie, Draguna
    Lewis, Frank L.
    [J]. IJCNN: 2009 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOLS 1- 6, 2009, : 2677 - 2684
  • [3] On Generalized Policy Iteration for Continuous-Time Linear Systems
    Lee, Jae Young
    Chun, Tae Yoon
    Park, Jin Bae
    Choi, Yoon Ho
    [J]. 2011 50TH IEEE CONFERENCE ON DECISION AND CONTROL AND EUROPEAN CONTROL CONFERENCE (CDC-ECC), 2011, : 1722 - 1728
  • [4] Explorized policy iteration for continuous-time linear systems
    Chun, Tae Yoon
    Choi, Yoon Ho
    Park, Jin Bae
    [J]. Transactions of the Korean Institute of Electrical Engineers, 2012, 61 (03): : 451 - 458
  • [5] Adaptive Optimal Control Algorithm for Continuous-Time Nonlinear Systems Based on Policy Iteration
    Vrabie, D.
    Lewis, F. L.
    [J]. 47TH IEEE CONFERENCE ON DECISION AND CONTROL, 2008 (CDC 2008), 2008, : 73 - 79
  • [6] Policy iteration for continuous-time systems with unknown internal dynamics
    Vrabie, D.
    Pastravanu, O.
    Lewis, F. L.
    [J]. 2007 MEDITERRANEAN CONFERENCE ON CONTROL & AUTOMATION, VOLS 1-4, 2007, : 34 - +
  • [7] Linear-Like Policy Iteration Based Optimal Control for Continuous-Time Nonlinear Systems
    Tahirovic, Adnan
    Astolfi, Alessandro
    [J]. IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2023, 68 (10) : 5837 - 5849
  • [8] Optimal Control for Continuous-time Nonlinear Systems based on a Linear-like Policy Iteration
    Tahirovic, Adnan
    Astolfi, Alessandro
    [J]. 2019 IEEE 58TH CONFERENCE ON DECISION AND CONTROL (CDC), 2019, : 5238 - 5243
  • [9] Policy Iteration Algorithm for Online Design of Robust Control for a Class of Continuous-Time Nonlinear Systems
    Wang, Ding
    Liu, Derong
    Li, Hongliang
    [J]. IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2014, 11 (02) : 627 - 632
  • [10] Continuous-Time Time-Varying Policy Iteration
    Wei, Qinglai
    Liao, Zehua
    Yang, Zhanyu
    Li, Benkai
    Liu, Derong
    [J]. IEEE TRANSACTIONS ON CYBERNETICS, 2020, 50 (12) : 4958 - 4971
12345