Modified Q-learning with distance metric and virtual target on path planning of mobile robot

被引:31
|
作者
Low, Ee Soong [1 ]
Ong, Pauline [1 ]
Low, Cheng Yee [1 ]
Omar, Rosli [2 ]
机构
[1] Univ Tun Hussein Onn Malaysia UTHM, Fac Mech & Mfg Engn, Batu Pahat 86400, Johor, Malaysia
[2] Univ Tun Hussein Onn Malaysia UTHM, Fac Elect & Elect Engn, Batu Pahat 86400, Johor, Malaysia
关键词
Moving target; Obstacle avoidance; Path planning; Q-learning; reinforcement learning; Mobile robot; ALGORITHM; OPTIMIZATION;
D O I
10.1016/j.eswa.2022.117191
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Path planning is an essential element in mobile robot navigation. One of the popular path planners is Q-learning - a type of reinforcement learning that learns with little or no prior knowledge of the environment. Despite the successful implementation of Q-learning reported in numerous studies, its slow convergence associated with the curse of dimensionality may limit the performance in practice. To solve this problem, an Improved Q-learning (IQL) with three modifications is introduced in this study. First, a distance metric is added to Q-learning to guide the agent moves towards the target. Second, the Q function of Q-learning is modified to overcome dead-ends more effectively. Lastly, the virtual target concept is introduced in Q-learning to bypass dead-ends. Experimental results across twenty types of navigation maps show that the proposed strategies accelerate the learning speed of IQL in comparison with the Q-learning. Besides, performance comparison with seven well-known path planners indicates its efficiency in terms of the path smoothness, time taken, shortest distance and total distance used.
引用
收藏
页数:40
相关论文
共 50 条
  • [1] Mobile Robot Path Planning using Q-Learning with Guided Distance and Moving Target Concept
    Low, Ee Soong
    Ong, Pauline
    Low, Cheng Yee
    [J]. INTERNATIONAL JOURNAL OF INTEGRATED ENGINEERING, 2021, 13 (02): : 177 - 188
  • [2] A Deterministic Improved Q-Learning for Path Planning of a Mobile Robot
    Konar, Amit
    Chakraborty, Indrani Goswami
    Singh, Sapam Jitu
    Jain, Lakhmi C.
    Nagar, Atulya K.
    [J]. IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2013, 43 (05): : 1141 - 1153
  • [3] Mobile robot path planning based on Q-learning algorithm
    Li, Shaochuan
    Wang, Xuiqing
    Hu, Liwei
    Liu, Ying
    [J]. 2019 WORLD ROBOT CONFERENCE SYMPOSIUM ON ADVANCED ROBOTICS AND AUTOMATION (WRC SARA 2019), 2019, : 160 - 165
  • [4] A Modified Q-learning Multi Robot Path Planning Algorithm
    Li, Bo
    Liang, Hongbin
    [J]. BASIC & CLINICAL PHARMACOLOGY & TOXICOLOGY, 2020, 127 : 125 - 126
  • [5] An optimized Q-Learning algorithm for mobile robot local path planning
    Zhou, Qian
    Lian, Yang
    Wu, Jiayang
    Zhu, Mengyue
    Wang, Haiyong
    Cao, Jinli
    [J]. KNOWLEDGE-BASED SYSTEMS, 2024, 286
  • [6] Extended Q-Learning Algorithm for Path-Planning of a Mobile Robot
    Goswami , Indrani
    Das, Pradipta Kumar
    Konar, Amit
    Janarthanan, R.
    [J]. SIMULATED EVOLUTION AND LEARNING, 2010, 6457 : 379 - +
  • [7] PATH PLANNING OF MOBILE ROBOT BASED ON THE IMPROVED Q-LEARNING ALGORITHM
    Chen, Chaorui
    Wang, Dongshu
    [J]. INTERNATIONAL JOURNAL OF INNOVATIVE COMPUTING INFORMATION AND CONTROL, 2022, 18 (03): : 687 - 702
  • [8] Dynamic Path Planning of a Mobile Robot with Improved Q-Learning algorithm
    Li, Siding
    Xu, Xin
    Zuo, Lei
    [J]. 2015 IEEE INTERNATIONAL CONFERENCE ON INFORMATION AND AUTOMATION, 2015, : 409 - 414
  • [9] Q-learning based method of adaptive path planning for mobile robot
    Li, Yibin
    Li, Caihong
    Zhang, Zijian
    [J]. 2006 IEEE INTERNATIONAL CONFERENCE ON INFORMATION ACQUISITION, VOLS 1 AND 2, CONFERENCE PROCEEDINGS, 2006, : 983 - 987
  • [10] Path planning of mobile robots with Q-learning
    Cetin, Halil
    Durdu, Akif
    [J]. 2014 22ND SIGNAL PROCESSING AND COMMUNICATIONS APPLICATIONS CONFERENCE (SIU), 2014, : 2162 - 2165