High-precision motion control of underwater gliders based on reinforcement learning

An underwater glider (UG) is a buoyancy-propelled and fixed-wing vehicle with attitude controlled completely by means of internal mass redistribution. Since the highly nonlinear and cross-coupled system dynamics, uncertain hydrodynamic parameters and uncertain internal dynamics, model-based control is hardly possible to employ in motion control of UGs. This paper presents a model-free motion control framework of UGs which named inverse model control (IMC). It takes the desired velocity as input, and outputs the control variables of directly and enables the glider to complete the behaviors of motion. Besides, we propose a novel method to represent the framework based on model-free reinforcement learning, which named heterogeneous agent asynchronous policy gradient (HAAPG). It has achieved high-precision motion control and could intermittently adjust the movable mass block rotation amount to correct the deviation caused by currents, which is energysaving. To verify the effectiveness of the IMC framework and the superiority of HAPPG algorithm, the dynamic model of underwater glider is established and a series of motion simulation scenarios are established. In addition, we conduct sea trials and deploy our IMC framework into actual underwater gliders to perform tasks and achieved satisfactory performance.