Ship Local Path Planning Based on Improved Q-Learning; [基于改进Q-Learning 的智能船舶局部路径规划]

The local path planning is an important part of the intelligent ship sailing in an unknown environment. In this paper, based on the reinforcement learning method of Q-Learning, an improved Q-Learning algorithm is proposed to solve the problems existing in the local path planning, such as slow convergence speed, high calculation complexity and easily falling into the local optimization. In the proposed method, the Q-table is initialized with respect to the artificial potential field, so that it has a prior knowledge of the environment. In addition, considering the heading factor of the ship, the two-dimensional position information is extended to the three-dimensional by joining the angle information. Then, the traditional reward function is modified by introducing the forward information and the obstacle information obtained by the sensor, and by adding the influence of the environment. Therefore, the proposed method is able to obtain the optimal path with the ship energy consumption reduced to a certain extent. The real-time capability and effectiveness of the algorithm are verified by the simulation and comparison experiments. © 2022, Editorial Board of Journal of Ship Mechanics. All right reserved.