Temporal Based Deep Reinforcement Learning for Crowded Lane Merging Maneuvers

In this paper, we propose a joint behavior and motion planning agent based on DRL (Deep Reinforcement Learning) intended for automated vehicles in a crowded merging scenario. The agent is trained using the PPO (Proximal Policy Optimization) algorithm, a state-of-the-art solution that ensures training stability and sample efficiency. We include temporal information in the observation of our agent to improve system stability. We have defined a simulated environment using the CARLA (Car Learning to Act) simulator, which handles the behavior of all other vehicles in the simulated world, in which the agent performs the merging maneuver. We perform a comparison between our temporal approach and an established, distance-based one, both in terms of safety, smoothness and comfort. We further analyze specific examples of both systems to describe the performance differences between them. Results show that our proposed agent yields a smoother, safer experience, and prove the viability of interweaving both systems within the same agent.