Online model-free reinforcement learning for the automatic control of a flexible wing aircraft

被引：12

作者：

Abouheaf, Mohammed ^{[1
,2
]}

Gueaieb, Wail ^{[1
]}

Lewis, Frank ^{[3
]}

机构：

[1] Univ Ottawa, Sch Elect Engn & Comp Sci, Ottawa, ON, Canada

[2] Aswan Univ, Coll Energy Engn, Elect Engn, Aswan, Egypt

[3] Univ Texas Arlington, Elect Engn, Arlington, TX 76019 USA

来源：

IET CONTROL THEORY AND APPLICATIONS | 2020年 / 14卷 / 01期

关键词：

dynamic programming; iterative methods; optimal control; Lyapunov methods; Riccati equations; aerodynamics; aerospace components; Markov processes; learning systems; approximation theory; learning (artificial intelligence); aircraft control; online model-free reinforcement learning; automatic control; flexible wing aircraft; prevailing high nonlinear deformations; flexible wing system; online control mechanism; reinforcement learning process; flexible wing aerial structures; model-free control policy framework; adaptive-critic mechanism; Bellman equation; asymptotically stable controller;

D O I：

10.1049/iet-cta.2018.6163

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

The control problem of the flexible wing aircraft is challenging due to the prevailing high non-linear deformations in the flexible wing system. This urged for new control mechanisms that are robust to the real-time variations in the wing's aerodynamics. An online control mechanism based on a value iteration reinforcement learning process is developed for flexible wing aerial structures. It employs a model-free control policy framework and a guaranteed convergent adaptive learning architecture to solve the system's Bellman optimality equation. A Riccati equation is derived and shown to be equivalent to solving the underlying Bellman equation. The online reinforcement learning solution is implemented using means of an adaptive-critic mechanism. The controller is proven to be asymptotically stable in the Lyapunov sense. It is assessed through computer simulations and its superior performance is demonstrated in two scenarios under different operating conditions.

引用

页码：73 / 84

页数：12

共 50 条

[31] Model-Free Decentralized Reinforcement Learning Control of Distributed Energy Resources
Mukherjee, Sayak
Bai, He
Chakrabortty, Aranya
[J]. 2020 IEEE POWER & ENERGY SOCIETY GENERAL MEETING (PESGM), 2020,
[32] Fusion of Microgrid Control With Model-Free Reinforcement Learning: Review and Vision
She, Buxin
Li, Fangxing
Cui, Hantao
Zhang, Jingqiu
Bo, Rui
[J]. IEEE TRANSACTIONS ON SMART GRID, 2023, 14 (04) : 3232 - 3245
[33] MODEL-FREE PREDICTIVE CONTROL OF NONLINEAR PROCESSES BASED ON REINFORCEMENT LEARNING
Shah, Hitesh
Gopal, M.
[J]. IFAC PAPERSONLINE, 2016, 49 (01): : 89 - 94
[34] Model-Free Reinforcement Learning based Lateral Control for Lane Keeping
Zhang, Qichao
Luo, Rui
Zhao, Dongbin
Luo, Chaomin
Qian, Dianwei
[J]. 2019 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2019,
[35] Learning Representations in Model-Free Hierarchical Reinforcement Learning
Rafati, Jacob
Noelle, David C.
[J]. THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FIRST INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / NINTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2019, : 10009 - 10010
[36] Model-Free Trajectory Optimization for Reinforcement Learning
Akrour, Riad
Abdolmaleki, Abbas
Abdulsamad, Hany
Neumann, Gerhard
[J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 48, 2016, 48
[37] Model-Free Active Exploration in Reinforcement Learning
Russo, Alessio
Proutiere, Alexandre
[J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
[38] Recovering Robustness in Model-Free Reinforcement Learning
Venkataraman, Harish K.
Seiler, Peter J.
[J]. 2019 AMERICAN CONTROL CONFERENCE (ACC), 2019, : 4210 - 4216
[39] Model-Free Reinforcement Learning Algorithms: A Survey
Calisir, Sinan
Pehlivanoglu, Meltem Kurt
[J]. 2019 27TH SIGNAL PROCESSING AND COMMUNICATIONS APPLICATIONS CONFERENCE (SIU), 2019,
[40] DATA-DRIVEN MODEL-FREE ITERATIVE LEARNING CONTROL USING REINFORCEMENT LEARNING
Song, Bing
Phan, Minh Q.
Longman, Richard W.
[J]. ASTRODYNAMICS 2018, PTS I-IV, 2019, 167 : 2579 - 2597

← 1 2 3 4 5 →