Stochastic Optimal Control of Unknown Linear Networked Control System using Q-Learning Methodology

被引：0
|
作者：
Xu, Hao [1 ]
Jagannathan, S. [1 ]
机构：
[1] Missouri Univ Sci & Technol, Dept Elect & Comp Engn, Rolla, MO 65409 USA
来源：
2011 AMERICAN CONTROL CONFERENCE | 2011年
关键词：
Networked Control System (NCS); Q-function; Adaptive Estimator (AE); Optimal Control;
D O I：
暂无
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
In this paper, the Bellman equation is utilized forward-in-time for the stochastic optimal control of Networked Control System (NCS) with unknown system dynamics in the presence of random delays and packet losses which are unknown. The proposed stochastic optimal control approach, referred normally as adaptive dynamic programming, uses an adaptive estimator (AE) and ideas from Q-learning to solve the infinite horizon optimal regulation control of NCS with unknown system dynamics. Update laws for tuning the unknown parameters of the adaptive estimator (AE) online to obtain the time-based Q-function are derived. Lyapunov theory is used to show that all signals are asymptotically stable (AS) and that the approximated control signals converge to optimal control inputs. Simulation results are included to show the effectiveness of the proposed scheme.
引用
收藏
页码：2819 / 2824
页数：6
相关论文
共 50 条
  • [1] Model-free H∞ Stochastic Optimal Design for Unknown Linear Networked Control System Zero-sum Games via Q-Learning
    Xu, Hao
    Jagannathan, S.
    2011 IEEE INTERNATIONAL SYMPOSIUM ON INTELLIGENT CONTROL (ISIC), 2011, : 198 - 203
  • [2] Stochastic Optimal CPS Relaxed Control Methodology for Interconnected Power Systems Using Q-Learning Method
    Yu, Tao
    Zhou, Bin
    Chan, Ka Wing
    Lu, En
    JOURNAL OF ENERGY ENGINEERING, 2011, 137 (03) : 116 - 129
  • [3] Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses
    Xu, Hao
    Jagannathan, S.
    Lewis, F. L.
    AUTOMATICA, 2012, 48 (06) : 1017 - 1030
  • [4] Off-policy Q-learning: Optimal tracking control for networked control systems
    Li J.-N.
    Yin Z.-X.
    Kongzhi yu Juece/Control and Decision, 2019, 34 (11): : 2343 - 2349
  • [5] Stochastic Inverse optimal control of unknown linear networked control system in the presence of random delays and packet losses
    Elvira-Ceja, Santiago
    Sanchez, Edgar N.
    Jagannathan, S.
    2015 AMERICAN CONTROL CONFERENCE (ACC), 2015, : 799 - 804
  • [6] Distributed Adaptive Optimal Regulation of Uncertain Large-Scale Linear Networked Control Systems Using Q-Learning
    Narayanan, Vignesh
    Jagannathan, S.
    2015 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (IEEE SSCI), 2015, : 587 - 592
  • [7] Optimal Control Inspired Q-Learning for Switched Linear Systems
    Chen, Hua
    Zheng, Linfang
    Zhang, Wei
    2020 AMERICAN CONTROL CONFERENCE (ACC), 2020, : 4003 - 4010
  • [8] Finite Horizon Stochastic Optimal Control of Uncertain Linear Networked Control System
    Xu, Hao
    Jagannathan, S.
    PROCEEDINGS OF THE 2013 IEEE SYMPOSIUM ON ADAPTIVE DYNAMIC PROGRAMMING AND REINFORCEMENT LEARNING (ADPRL), 2013, : 24 - 30
  • [9] Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics
    Kiumarsi, Bahare
    Lewis, Frank L.
    Modares, Hamidreza
    Karimpour, Ali
    Naghibi-Sistani, Mohammad-Bagher
    AUTOMATICA, 2014, 50 (04) : 1167 - 1175
  • [10] Reinforcement Q-Learning for PDF Tracking Control of Stochastic Systems with Unknown Dynamics
    Yang, Weiqing
    Zhou, Yuyang
    Zhang, Yong
    Ren, Yan
    MATHEMATICS, 2024, 12 (16)
12345