Stochastic Optimal Control of Unknown Linear Networked Control System using Q-Learning Methodology

被引：0

作者：

Xu, Hao ^{[1
]}

Jagannathan, S. ^{[1
]}

机构：

[1] Missouri Univ Sci & Technol, Dept Elect & Comp Engn, Rolla, MO 65409 USA

来源：

2011 AMERICAN CONTROL CONFERENCE | 2011年

关键词：

Networked Control System (NCS); Q-function; Adaptive Estimator (AE); Optimal Control;

D O I：

暂无

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

In this paper, the Bellman equation is utilized forward-in-time for the stochastic optimal control of Networked Control System (NCS) with unknown system dynamics in the presence of random delays and packet losses which are unknown. The proposed stochastic optimal control approach, referred normally as adaptive dynamic programming, uses an adaptive estimator (AE) and ideas from Q-learning to solve the infinite horizon optimal regulation control of NCS with unknown system dynamics. Update laws for tuning the unknown parameters of the adaptive estimator (AE) online to obtain the time-based Q-function are derived. Lyapunov theory is used to show that all signals are asymptotically stable (AS) and that the approximated control signals converge to optimal control inputs. Simulation results are included to show the effectiveness of the proposed scheme.

引用

页码：2819 / 2824

页数：6

共 50 条

[31] On-policy Q-learning for Adaptive Optimal Control
Jha, Sumit Kumar
Bhasin, Shubhendu
2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), 2014, : 301 - 306
[32] Input-Decoupled Q-Learning for Optimal Control
Minh Q. Phan
Seyed Mahdi B. Azad
The Journal of the Astronautical Sciences, 2020, 67 : 630 - 656
[33] Input-Decoupled Q-Learning for Optimal Control
Phan, Minh Q.
Azad, Seyed Mahdi B.
JOURNAL OF THE ASTRONAUTICAL SCIENCES, 2020, 67 (02): : 630 - 656
[34] Q-Learning Methods for LQR Control of Completely Unknown Discrete-Time Linear Systems
Fan, Wenwu
Xiong, Junlin
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2025, 22 : 5933 - 5943
[35] An ADDHP-based Q-learning algorithm for optimal tracking control of linear discrete-time systems with unknown dynamics
Mu, Chaoxu
Zhao, Qian
Sun, Changyin
Gao, Zhongke
APPLIED SOFT COMPUTING, 2019, 82
[36] Networked Control of Nonlinear Systems under Partial Observation Using Continuous Deep Q-Learning
Ikemoto, Junya
Ushio, Toshimitsu
2019 IEEE 58TH CONFERENCE ON DECISION AND CONTROL (CDC), 2019, : 6793 - 6798
[37] Stochastic linear quadratic optimal control for model-free discrete-time systems based on Q-learning algorithm
Wang, Tao
Zhang, Huaguang
Luo, Yanhong
NEUROCOMPUTING, 2018, 312 : 1 - 8
[38] Based on Q-Learning Optimal Tracking Control Schemes for Linear It(O)over-cap Stochastic Systems With Markovian Jumps
Li, Mei
Sun, Jiayue
Zhang, Huaguang
Ming, Zhongyang
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS, 2023, 70 (03) : 1094 - 1098
[39] Adaptive Optimal Control via Continuous-Time Q-Learning for Unknown Nonlinear Affine Systems
Chen, Anthony Siming
Herrmann, Guido
2019 IEEE 58TH CONFERENCE ON DECISION AND CONTROL (CDC), 2019, : 1007 - 1012
[40] Q-Learning Based Optimal Tracking Control of Free-Flying Space Manipulators with Unknown Dynamics
Zhu, Hongxu
Wu, Shufan
Shen, Qiang
Sun, Ran
2022 41ST CHINESE CONTROL CONFERENCE (CCC), 2022, : 2357 - 2362

← 1 2 3 4 5 →