A learning method for AUV collision avoidance through deep reinforcement learning

This paper investigates the collision avoidance problem for autonomous underwater vehicle (AUV) with limited detection range via deep reinforcement learning method in an unknown underwater environment. Firstly, to deal with both unknown static and dynamic obstacles, we introduce two different event-triggered mechanism to generate obstacle-related states and reward the safe collision avoidance action of AUV, respectively. Secondly, the complete state space is customized by combining the state of AUV itself and the state related to a target area. In addition, considering the task requirement of reaching a target area with the shortest path, the complete reward function is designed. Then, we propose a novel event-triggered soft actor-critic (ET-SAC) algorithm for AUV collision avoidance, and give its detailed pseudo code and implementation architecture. Finally, training and evaluation are carried out in the simulation platform we designed, including unknown static obstacle environment and unknown dynamic obstacle environment in 2D/3D space, and the results confirm that the proposed algorithm of this paper is feasible and effective for AUV to avoid collision safely.