Merging in Congested Freeway Traffic Using Multipolicy Decision Making and Passive Actor-Critic Learning

Merging in congested freeway traffic is a significant challenge toward realizing fully automated (level 4) driving. Merging vehicles need to decide not only how to merge safely into a spot, but also where to merge. We present a method for freeway merge based on multipolicy decision making coupled with a reinforcement learning technique called passive actor-critic (pAC), which learns with less knowledge of the system and without active exploration. The multipolicy decision making selects a candidate spot formerging by using the state value learned by pAC. Together, these techniques yield a method that first decides where to merge and then realizes safe merging. We evaluate our method using real traffic data. Our experiments show that pAC achieves an overall success rate of 92% for merging into a predetermined spot on a freeway, which is comparable to human decision making.