Enhancing air traffic control: A transparent deep reinforcement learning framework for autonomous conflict resolution

Rising traffic demands, advancements in automation, and communication breakthroughs are opening new design frontiers for future air traffic controllers (ATCs). This article introduces a deep reinforcement learning (DRL) controller to support conflict resolution in autonomous free flight. While DRL has made significant strides in this domain, there is scant focus in existing research on the explainability and safety of DRL controllers, especially their robustness against adversarial attacks. To tackle these concerns, we have engineered a fully transparent DRL framework that: (i) separates the intertwined Q-value learning model into distinct modules for safety-awareness and efficiency (target attainment); and (ii) incorporates data from nearby intruders directly, thereby obviating the need for central control. Our simulated experiments demonstrate that this bifurcation of safety-awareness and efficiency not only enhances performance in free flight control tasks but also substantially boosts practical explainability. Moreover, the safety-oriented Q-learning component offers detailed insights into environmental risk factors. To assess resilience against adversarial attacks, we present a novel attack strategy that induces both safety-centric and efficiency-centric disruptions. This adversary aims to degrade safety/efficiency by targeting the agent at select time intervals. Our experimental results reveal that this targeted approach can provoke as many collisions as a uniform attack strategy — that is, attacking at every opportunity — while engaging the agent four times less frequently, shedding light on the potential and limitations of DRL in future ATC system designs. The source code is available to the public at https://github.com/WLeiiiii/Gym-ATC-Attack-Project. © 2024