Hybrid deep reinforcement learning based eco-driving for low-level connected and automated vehicles along signalized corridors

Eco-Driving has great potential in reducing the fuel consumption of road vehicles, especially under the connected and automated vehicles (CAVs) environment. Traditional model-based Eco-Driving methods usually require sophisticated models and cannot deal with complex driving scenarios. This paper proposes a hybrid reinforcement learning (RL) based Eco-Driving algorithm considering both the longitudinal acceleration/deceleration and the lateral lane-changing operations. A deep deterministic policy gradient (DDPG) algorithm is designed to learn the continuous longitudinal acceleration/deceleration to reduce the fuel consumption as well as to maintain acceptable travel time. Collecting the critic's value of each single lane from DDPG and integrating the information of adjacent lanes, a deep Q-learning algorithm is developed to make the discrete lane-changing decision. Together, a hybrid deep Q-learning and policy gradient (HDQPG) method is developed for vehicles driving along multi-lane urban signalized corridors. The method can enable the controlled vehicle to learn well-established longitudinal fuel-saving strategies, and to perform appropriate lane-changing operations at proper times to avoid congested lanes. Numerical experiments show that HDQPG can reduce fuel consumption by up to 46% with marginal or no increase of travel times.