Speed and heading control of an unmanned surface vehicle using deep reinforcement learning

In this paper, a deep reinforcement learning-based speed and heading control method is proposed for an unmanned surface vehicle (USV). A deep deterministic policy gradient (DDPG) algorithm which combines with an actor-critic reinforcement learning mechanism, is adopted to provide continuous control variables by interacting with the environment. Moreover, two types of reward functions are created for speed and heading control of the USV. The control policy is trained by trial and error so that the USV can be guided to achieve the desired speed and heading angle steadily and rapidly. Simulation results verify the feasibility and effectiveness of the proposed approach by comparisons with classical PID control and S plane control.