Adaptive Path Planning for Autonomous Ships Based on Deep Reinforcement Learning Combined with Images

This paper presents a proximal policy optimization with route guidance (PPORG) algorithm for the autonomous ships for collision avoidance and path planning. The PPORG algorithm creates an image route guidance method based on deep reinforcement learning to make the agent not deviate from the target area. The collision avoidance process is modeled as a partially observable Markov decision process (POMDP) for the ship impossible to obtain the information of various obstacles in the uncertain environments before path planning. The model uses images as the state input to adaptively obtain all the local information and designs a segment reward function to realize the adaptive path planning for different routes. Thanks to deep reinforcement learning, the PPORG can adaptively achieve path planning for different routes with obstacles. Compared to traditional deep reinforcement learning methods, the PPORG can provide better decisions. We demonstrate the performance of the PPORG using different static obstacle scenarios.