Intelligent Mission Supervisor Design for Null-space-based Behavioral Control System: A Reinforcement Learning Approach

In this paper, the intelligent mission supervisor for null-space-based behavioral control (NSBC) is developed. A novel reinforcement learning (RL) based approach is proposed to overcome the lack ability of the mission priority adjustment in the classic NSBC methodologies. The traditional mission supervisor design of NSBC rely on human's ability to determine every individual mission's priority in advance. By elaborately combining the NSBC concept and the Deep Q Network (DQN) method, we develops a new mission management approach which can adjust the agent's behavior priorities dynamically. This approach is implemented without manual design rules and avoiding online calculations after offline training. In order to achieve better learning effect and faster network convergence performance, the experience replay and network structure are optimized by applying priority replay and dueling structure, respectively. Finally, simulation results show that the proposed reinforcement learning mission supervisor (RLMS) can provide better mission priority adjustment performance than the traditional mission supervisor using a much lower sampling frequency and much fewer numbers of dynamic mission priority adjustment.