Reinforcement learning based robust tracking control for unmanned helicopter with state constraints and input saturation

In this paper, an online adaptive optimal control scheme using reinforcement learning (RL) methodology is developed with applications to helicopters in the presence of input saturation and state constraints. Such a control scheme can overcome the strong nonlinearity and coupling dynamics of helicopters by deploying adaptive critic designs (ACDs). Firstly, the backstepping technique is employed to divide the helicopter system into a kinematic loop and a dynamic loop. In the kinematic loop, a constrained Hamilton-Jacobi-Bellman (HJB) equation containing a barrier function is designed to satisfy state constraints. In the dynamic loop, an input- dependent non-quadratic term is incorporated into the HJB equation to solve the input-constrained optimal control problem. Then, a radial basis function (RBF) neural network (NN) is introduced to establish actor-critic networks for the implementation of adaptive optimal control. The critic network is exploited to optimize the tracking performance, while the approximated optimal control for the nominal error dynamic model is derived from the actor network. Meanwhile, a disturbance observer based on RBF NN is designed to compensate for uncertain system dynamics and external disturbances. Using the concurrent learning technique, a novel online update law of actor-critic networks is designed to relax the persistence of excitation (PE) condition. Moreover, the uniform ultimate boundedness (UUB) of parameter estimation error and the asymptotic convergence of state tracking errors are proven through the Lyapunov-based stability analysis. Finally, simulation results are presented to demonstrate that the proposed control strategy is suitable and effective for the helicopter attitude and altitude tracking control problem.