6-DOF Reinforcement Learning Control for Multi-rotor and Fixed-Wing Aircrafts

Autonomous flight control is a key technology for Unmanned Aerial Vehicles (UAVs). The model-based controller design approaches usually aim at concrete vehicles, and the controllers developed lack universality. Reinforcement learning provides a general controller design paradigm that is adaptive, optimal, model-free and widely applicable. It is a promising way to realize the intelligent control, allowing for easy automation of the controller design. Compared with the 3 Degree-of-Freedom (DOF) flight, the 6 DOF model accurately characterizes the real flight of aircrafts while the implementation of intelligent control is more difficult. Based on the Deep Reinforcement Learning (DRL) for the multidimensional continuous state inputs and continuous action outputs problems, this paper summarizes the 6 DOF reinforcement learning controller design for two typical aircrafts of multi-rotor and fixed-wing, including the hovering maneuver control and forward flight control for the quadrotor, and the cruise control and the Cobra post-stall maneuver control for the fixed-wing aircraft. It is shown that the intelligent controllers learn the control law of specific tasks without the model information, and they perform well with desired flexibility and robustness. The resulting end-to-end intelligent controller may realize the integrated optimal control, avoiding the traditional dual-loop structure regarding the trajectory and attitude control. This is advantageous to the exploitation of the nonlinear effects including the inertia coupling and the aerodynamics.