End-to-End Urban Autonomous Driving With Safety Constraints

End-to-end autonomous driving systems aim to address the perception, decision-making, and control challenges in a self-contained way, facilitating easier adaptation to new scenarios and improved scalability. However, existing end-to-end approaches often lack safety constraints and are limited to simple driving tasks. In this study, we propose a novel method that integrates safety constraints into deep reinforcement learning for end-to-end autonomous driving, capable of handling complex urban scenarios. To achieve this, we introduce safety information into the probabilistic graphical model(PGM) and learn it in conjunction with the reinforcement learning process. An auxiliary safety critic evaluates the safety performance of the ego vehicle and assists in training the driving policy. Through the safety critic, the learned policy is able to provide better actions to control the ego vehicle during task completion. A sequential latent environment model is introduced and learned jointly with the reinforcement learning process. Compared to existing methods, our approach not only incorporates predefined safety constraints but also offers end-to-end training with enhanced interpretability through the latent model. This interpretability provides insights into decision-making and aids in debugging. We evaluate our algorithm using the CARLA simulator for autonomous vehicles, demonstrating its effectiveness in handling complex urban scenarios and providing enhanced safety information, even in crowded environments with multiple surrounding vehicles. The code (https://github.com/houchangmeng/safe-e2e-driving) will be made publicly available after the article is published.