Distributional Soft Actor-Critic-Based Multi-AUV Cooperative Pursuit for Maritime Security Protection

Unauthorized underwater vehicles (UUVs) pose a serious threat to maritime security. To preserve maritime security, it is essential to pursue these UUVs. The majority of traditional pursuit methods are based on known environmental dynamics. However, the underwater environment is too complicated and unpredictable to describe these dynamics accurately. This study developed a novel online decision-making technique called multi-agent distributional soft actor-critic (MADA) to handle the issue of underwater cooperative pursuit. The method constructs a control-oriented framework based on multi-agent reinforcement learning that can map autonomous underwater vehicle (AUV) observations to pursuit actions. Multiple AUVs can combine to make prompt pursuit decisions. Then, the proposed method combines distributional soft actor-critic and curriculum learning to improve the success rates of multiple AUVs in pursuing UUVs. Experimental results show that the MADA can obtain a better cooperative pursuit strategy.