Decision-making for the autonomous navigation of USVs based on deep reinforcement learning under IALA maritime buoyage system

It is extremely challenging to carry out an advanced adaptive navigation system in the complex environment of a port. Especially in the waters that are about to be berthed, the unmanned surface vehicles (USVs) should autonomously identify and avoid dense buoys according to the rules. Each country has corresponding mandatory documents that strictly regulate the shape, color and orientation of water buoys in coastal and estuary channels. Aiming at the problem of USV collision avoidance navigation under the constraints of the IALA (International Association of Marine Aids and Lighthouse Authorities) Maritime buoyage system, this paper designs a mixed environment model including multiple ocean buoys according to this rule. By using the deep neural network to extract the state features of the target, and setting the reward function reasonably, the USV can not only navigate to the target autonomously, but also identify the corresponding buoy and give the corresponding decision. Using different DQN optimization algorithms to conduct comparative experiments, the stability of the algorithm's learning of the optimal strategy is improved. The results show that the algorithm can accurately avoid obstacles ahead, identify buoys effectively, and realize effective autonomous collision avoidance decision-making in complex environments with static obstacles and buoys with indicating function. This research can provide theoretical basis and method reference for USV's autonomous navigation in port.