Autonomous Decision Making of UAV in Short-Range Air Combat Based on DQN Aided by Expert Knowledge

Recently, reinforcement learning (RL) has emerged in the field of autonomous air combat. However, it is well known that RL has the problems of low exploration efficiency and long training time in practical application. In this paper, we propose autonomous maneuver decision model based on deep Q-learning network (DQN) incorporating expert knowledge. First, we design a series of exploration rules based on expert knowledge. With the help of exploration rules, UAV is no longer randomly exploring in the whole space, but is able to avoid ineffective space exploration to improve exploration efficiency. In addition, we also introduce Imitation Learning (IL) to obtain an initial strategy for RL from the decision trajectory data demonstrated by human experts, which can speed up the training process. Finally, the simulation results verify the effectiveness of the UAV autonomous maneuver decision model.