Design of a Reinforcement Learning PID controller

被引:11
|
作者
Guan, Zhe [1 ]
Yamamoto, Tom [2 ]
机构
[1] Hiroshima Univ, Dream Driven Cocreat Res Ctr, KOBELCO Construct Machinery, Higashihiroshima, Japan
[2] Hiroshima Univ, Grad Sch Adv Sci & Engn, Higashihiroshima, Japan
关键词
Adaptive control; PID control; Reinforcement Learning; NETWORKS;
D O I
10.1109/ijcnn48605.2020.9207641
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This paper addresses a design problem of a Proportional-Integral-Derivative (PID) controller with new adaptive updating rule based on Reinforcement Learning (RL) approach for nonlinear systems. A new design scheme that RL can be used to complement the conventional control technology PID is presented. In this study, a single Radial Basis Function (RBF) network is introduced to calculate the control policy function of Actor and the value function of Critic simultaneously. Regarding to the PID controller structure, the inputs of RBF network are system error, the difference of output as well as the second order difference of output, and they are defined as system states. The Temporal Difference (TD) error in this study is newly defined and involves the error criterion which is defined by the difference between one-step ahead prediction and the reference value. The gradient descent method is adopted based on TD error performance index, then the updating rules can be obtained. Therefore, the network weights and the kernel function can be calculated in an adaptive manner. Finally, the numerical simulations are conducted in nonlinear systems to illustrate the efficiency and robustness of the proposed scheme.
引用
收藏
页数:6
相关论文
共 50 条
  • [1] Design of a reinforcement learning PID controller
    Guan, Zhe
    Yamamoto, Toru
    [J]. IEEJ TRANSACTIONS ON ELECTRICAL AND ELECTRONIC ENGINEERING, 2021, 16 (10) : 1354 - 1360
  • [2] Reinforcement learning based PID controller design for LFC in a microgrid
    Esmaeili, Mehran
    Shayeghi, Hossein
    Nejad, Hamid Mohammad
    Younesi, Abdollah
    [J]. COMPEL-THE INTERNATIONAL JOURNAL FOR COMPUTATION AND MATHEMATICS IN ELECTRICAL AND ELECTRONIC ENGINEERING, 2017, 36 (04) : 1287 - 1297
  • [3] Improve PID controller through reinforcement learning
    Qin, Yunxiao
    Zhang, Weiguo
    Shi, Jingping
    Liu, Jinglong
    [J]. 2018 IEEE CSAA GUIDANCE, NAVIGATION AND CONTROL CONFERENCE (CGNCC), 2018,
  • [4] Design of ABR Flow Controller Based on Reinforcement Learning-PID Method
    Zhao, Xin
    Li, Xin
    [J]. 2008 CHINESE CONTROL AND DECISION CONFERENCE, VOLS 1-11, 2008, : 397 - +
  • [5] Intelligent PID Controller Based on Deep Reinforcement Learning
    Zhai, Yinhe
    Zhao, Qiang
    Han, Yinghua
    Wang, Jinkuan
    Zeng, Wenying
    [J]. 2024 8TH INTERNATIONAL CONFERENCE ON ROBOTICS, CONTROL AND AUTOMATION, ICRCA 2024, 2024, : 343 - 348
  • [7] Fuzzy PID Controller for UAV Based on Reinforcement Learning
    Zhang, Benyi
    Zhang, Weiping
    Mou, Jiawang
    Yang, Runmin
    Zhang, Yichen
    [J]. PROCEEDINGS OF 2022 INTERNATIONAL CONFERENCE ON AUTONOMOUS UNMANNED SYSTEMS, ICAUS 2022, 2023, 1010 : 1724 - 1732
  • [8] A Proposal of Adaptive PID Controller Based on Reinforcement Learning
    WANG, Xue-song
    CHENG, Yu-hu
    SUN, Wei
    [J]. Journal of China University of Mining and Technology, 2007, 17 (01): : 40 - 44
  • [9] Reinforcement learning based adaptive PID controller design for control of linear/nonlinear unstable processes
    Shuprajhaa, T.
    Sujit, Shiva Kanth
    Srinivasan, K.
    [J]. APPLIED SOFT COMPUTING, 2022, 128
  • [10] Combination continuous action reinforcement learning automata & PSO for design of PID controller for AVR system
    Hashemi, F.
    Mohammadi, M.
    [J]. International Journal of Engineering, Transactions A: Basics, 2015, 28 (01): : 54 - 61