Prioritized experience replay based reinforcement learning for adaptive tracking control of autonomous underwater vehicle

A novel adaptive trajectory tracking control method is proposed in this paper for au- tonomous underwater vehicle systems with external disturbances. The control action is composed of sliding mode control and action-depended heuristic dynamic programming (ADHDP) controller to track the expected sailing position and angle in the horizontal co- ordinate system with fixed depth. As an auxiliary control of sliding mode control, the AD- HDP controller observes the difference between the actual sailing position/angle and the expected sailing position/angle, and adaptively provides corresponding supplementary con- trol actions using a data-driven fashion. At the same time, we design a weight-related pri- ority experience replay (PER) technology to update the online weight network by using the relevant historical data stored in the database to improve the learning rate. The proposed algorithm can adjust parameters online under various conditions. Furthermore, it is very suitable for underwater vehicle systems with parameter uncertainties and external distur- bances. Based on Lyapunov stability method, the stability of closed-loop system state and network weight error is analyzed. Finally, the validity of our control strategy is verified by simulation.(c) 2022 Elsevier Inc. All rights reserved.