Multi-layer Control Architecture for Unsignalized Intersection Management via Nonlinear MPC and Deep Reinforcement Learning

In this study, a multi-layer control architecture for unsignalized intersection management is proposed. The architecture is composed of two layers. The low-level layer deals with a single decoupled vehicle model controlled depending on a dynamical-based Nonlinear Model Predictive Control (NMPC) algorithm. The high-level layer holds the decision-making module which is controlled via a centralized-trained/ decentralized-executed Multi Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) algorithm. In the proposed architecture, the high-level decision making layer generates a reference velocity for each vehicle to pass the unsignalized intersection without collisions. The NMPC is utilized to ensure that each vehicle follows the provided trajectory, taking into account the vehicle's acceleration profile to attain passengers' comfort. The proposed approach is trained and executed in a custom-built multi-agent intersection environment simulated using CARLA simulator. Results indicate effective training as the MATD3 was able to encounter the suitable policy to avoid the intersection collisions with a success percentage of 100%.