Adaptive PID controller based on Q-learning algorithm

被引:29
|
作者
Shi, Qian [1 ]
Lam, Hak-Keung [1 ]
Xiao, Bo [2 ]
Tsai, Shun-Hung [3 ]
机构
[1] Kings Coll London, Dept Informat, Bush House,Strand Campus,30 Aldwych, London WC2B 4BG, England
[2] Imperial Coll London, Hamlyn Ctr Robot Surg, London SW7 2AZ, England
[3] Natl Taipei Univ Technol, Grad Inst Automat Technol, Taipei 10608, Taiwan
关键词
D O I
10.1049/trit.2018.1007
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
An adaptive proportional-integral-derivative (PID) controller based on Q-learning algorithm is proposed to balance the cart-pole system in simulation environment. This controller was trained using Q-learning algorithm and implemented the learned Q-tables to change the gains of linear PID controllers according to the state of the system during the control process. The adaptive PID controller based on Q-learning algorithm was trained from a set of fixed initial positions and was able to balance the system starting from a series of initial positions that are different from the ones used in the training session, which achieved equivalent or even better performances in comparison with the conventional PID controller and the controller only uses Q-learning algorithm. This indicates the advantage of the adaptive PID controller based on Q-learning algorithm both in the generality of balancing the cart-pole system from a relatively wide range of initial positions and in the stabilisability of achieving smaller steady-state error.
引用
收藏
页码:235 / 244
页数:10
相关论文
共 50 条
  • [1] Fuzzy PID Controller Design Using Q-Learning Algorithm with a Manipulated Reward Function
    Aghaei, Vahid Tavakol
    Onat, Ahmet
    Eksin, Ibrahim
    Guzelkaya, Mujde
    [J]. 2015 EUROPEAN CONTROL CONFERENCE (ECC), 2015, : 2502 - 2507
  • [2] Voltage Control of Power System employing Q-Learning based PID Controller
    Saha, Jayita
    Guha, Dipayan
    Jha, Sumit Kumar
    [J]. 2022 IEEE 19TH INDIA COUNCIL INTERNATIONAL CONFERENCE, INDICON, 2022,
  • [3] Q-LEARNING BASED CONTROL ALGORITHM FOR HTTP ADAPTIVE STREAMING
    Martin, Virginia
    Cabrera, Julian
    Garcia, Narciso
    [J]. 2015 VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP), 2015,
  • [4] Q-Learning Based Distributed Adaptive Algorithm for Topological Stability
    Huang, Qing-Dong
    Shi, Bin-Yu
    Guo, Min-Peng
    Yuan, Run-Zhi
    Chen, Chen
    [J]. Dianzi Keji Daxue Xuebao/Journal of the University of Electronic Science and Technology of China, 2020, 49 (02): : 262 - 268
  • [5] Q-learning Based Adaptive PID Controller Design for AMT Clutch Engagement During Start-up Process
    Lu Xiaohui
    Gao Bingzhao
    Chen Hong
    [J]. PROCEEDINGS OF THE 31ST CHINESE CONTROL CONFERENCE, 2012, : 3131 - 3136
  • [6] Design and Applications of Q-Learning Adaptive PID Algorithm for Maglev Train Levitation Control System
    Shou, Baineng
    Zhang, Hehong
    Long, Zhiqiang
    Xie, Yunde
    Zhang, Ke
    Gu, Qiuming
    [J]. 2023 35TH CHINESE CONTROL AND DECISION CONFERENCE, CCDC, 2023, : 1947 - 1953
  • [7] Adaptive sensor-planning algorithm with Q-learning
    Maeda, M
    Kato, N
    Kashimura, H
    [J]. 2004 IEEE CONFERENCE ON CYBERNETICS AND INTELLIGENT SYSTEMS, VOLS 1 AND 2, 2004, : 966 - 969
  • [8] Incremental Q-learning strategy for adaptive PID control of mobile robots
    Carlucho, Ignacio
    De Paula, Mariano
    Villar, Sebastian A.
    Acosta, Gerardo G.
    [J]. EXPERT SYSTEMS WITH APPLICATIONS, 2017, 80 : 183 - 199
  • [9] Adaptive Traffic Control Algorithm Based on Back-Pressure and Q-Learning
    Maipradit, Arnan
    Gao, Juntao
    Kawakami, Tomoya
    Ito, Minoru
    [J]. 2019 IEEE INTELLIGENT TRANSPORTATION SYSTEMS CONFERENCE (ITSC), 2019, : 1995 - 1999
  • [10] Adaptive job shop scheduling strategy based on weighted Q-learning algorithm
    Wang, Yu-Fang
    [J]. JOURNAL OF INTELLIGENT MANUFACTURING, 2020, 31 (02) : 417 - 432