Deep reinforcement learning based collision avoidance system for autonomous ships

Autonomous ships is a key to avoid accidents caused by human errors and improve maritime safety. However, unlike the autonomous vehicles counterpart, collision avoidance for autonomous ships faces many challenges due to the hash driving environments, difficult ship control and large stopping distance. In this paper, we investigate a collision avoidance system for autonomous ships under complex encounter scenarios, such as busy ports. In the system various sensors are used to detect objects and perceive the maritime environments. To help the autonomous ships handle the complex and dynamic scenarios that may be encountered, a collision map used to describe the ships encounter scenarios is generated and utilized as the input of a deep reinforcement learning (DRL) model. The DRL model is applied to make collision avoidance and safe driving decisions. New reward functions are proposed to train the DRL model to generate safe ship maneuver actions to reduce collisions and ensure compliance with the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs). Furthermore, a self-adaptive parameters sharing approach is designed for fast convergence and collision avoidance performance of the DRL model, where the parameters of the fully connected layers are shared and the correlation layers are self adapted for the DRL critic and actor networks. Simulation results show that the proposed system has high DRL convergence speed and excellent collision avoidance.