An end-to-end learning of driving strategies based on DDPG and imitation learning

被引:0
|
作者
Zou, Qijie [1 ,2 ]
Xiong, Kang [1 ]
Hou, Yingli [1 ]
机构
[1] Dalian Univ, Informat Engn Coll, Dalian 116000, Liaoning, Peoples R China
[2] Natl Innovat Inst Def Technol, Unmanned Syst Res Ctr, Changsha 410000, Human, Peoples R China
基金
中国国家自然科学基金;
关键词
driving strategy; imitation learning; deep reinforcement learning; DDPG; experience separation;
D O I
暂无
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The Deep Deterministic Policy Gradient Algorithm (DDPG) has great advantages in continuous control problems and plays a very important role in the field of autonomous driving. However, the performance of the traditional DDPG algorithm depends on the initialization parameter settings, and it is difficult to achieve satisfactory results in the actual environment. At the same time, traditional DDPG also needs a lot of exploration to converge to a suitable control strategy. This paper proposes a DDPG algorithm framework based on imitation learning (DDPG-IL). The framework first obtains demonstration data (via IL) and stores it in the expert pool, meanwhile the DDPG algorithm is pre-trained. Then the algorithm makes reasonable use of the demonstration data and its own exploration data for learning. Finally, when the algorithm reaches an approximate expert level, it gradually becomes an ordinary reinforcement learning and continues training by self-learning until the algorithm converges to a stable state. The experimental comparison on the racing simulator TORCS proves that our proposed DDPG-IL algorithm has more advantages and can obtain better performance than the traditional DDPG algorithm.
引用
收藏
页码:3190 / 3195
页数:6
相关论文
共 50 条
  • [1] End-to-end Driving via Conditional Imitation Learning
    Codevilla, Felipe
    Mueller, Matthias
    Lopez, Antonio
    Koltun, Vladlen
    Dosovitskiy, Alexey
    [J]. 2018 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2018, : 4693 - 4700
  • [2] Hierarchical Interpretable Imitation Learning for End-to-End Autonomous Driving
    Teng, Siyu
    Chen, Long
    Ai, Yunfeng
    Zhou, Yuanye
    Xuanyuan, Zhe
    Hu, Xuemin
    [J]. IEEE TRANSACTIONS ON INTELLIGENT VEHICLES, 2023, 8 (01): : 673 - 683
  • [3] End-to-End Deep Conditional Imitation Learning for Autonomous Driving
    Abdou, Mohammed
    Kamal, Hanan
    El-Tantawy, Samah
    Abdelkhalek, Ali
    Adel, Omar
    Hamdy, Karim
    Abaas, Mustafa
    [J]. 31ST INTERNATIONAL CONFERENCE ON MICROELECTRONICS (IEEE ICM 2019), 2019, : 346 - 350
  • [4] Query-Efficient Imitation Learning for End-to-End Simulated Driving
    Zhang, Jiakai
    Cho, Kyunghyun
    [J]. THIRTY-FIRST AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2017, : 2891 - 2897
  • [5] Agile Autonomous Driving using End-to-End Deep Imitation Learning
    Pan, Yunpeng
    Cheng, Ching-An
    Saigol, Kamil
    Lee, Keuntaek
    Yan, Xinyan
    Theodorou, Evangelos A.
    Boots, Byron
    [J]. ROBOTICS: SCIENCE AND SYSTEMS XIV, 2018,
  • [6] Generative Adversarial Imitation Learning for End-to-End Autonomous Driving on Urban Environments
    Karl Couto, Gustavo Claudio
    Antonelo, Eric Aislan
    [J]. 2021 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (IEEE SSCI 2021), 2021,
  • [7] End-to-End Differentiable Adversarial Imitation Learning
    Baram, Nir
    Anschel, Oron
    Caspi, Itai
    Mannor, Shie
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 70, 2017, 70
  • [8] Bayesian Imitation Learning for End-to-End Mobile Manipulation
    Du, Yuqing
    Ho, Daniel
    Alemi, Alexander A.
    Jang, Eric
    Khansari, Mohi
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162, 2022,
  • [9] A Survey on Imitation Learning Techniques for End-to-End Autonomous Vehicles
    Le Mero, Luc
    Yi, Dewei
    Dianati, Mehrdad
    Mouzakitis, Alexandros
    [J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2022, 23 (09) : 14128 - 14147
  • [10] End-to-End Autonomous Driving Decision Based on Deep Reinforcement Learning
    Huang, Zhiqing
    Zhang, Ji
    Tian, Rui
    Zhang, Yanxin
    [J]. CONFERENCE PROCEEDINGS OF 2019 5TH INTERNATIONAL CONFERENCE ON CONTROL, AUTOMATION AND ROBOTICS (ICCAR), 2019, : 658 - 662