Mitigating Bus Bunching via Hierarchical Multi-Agent Reinforcement Learning

Bus bunching is harmful to the efficiency and stability of bus transit systems, consequently delaying the arrival time of passengers and lowering the public transportation's adoption rate. Traditional solutions adjust the additional holding time of buses at certain stations to mitigate this phenomenon. These methods sacrifice the system efficiency in exchange for even headway between neighboring buses. Little work focuses on optimizing multiple strategies when a single bus line not only has a bus bay to increase bus dwell time but also owns several dedicated bus lanes to accelerate. In this work, we develop a hierarchical multi-agent reinforcement learning (HMARL) framework to combine these two strategies. Speeding up certain buses via dedicated lanes can counteract the negative influence of additional holding time. Next, to support the two strategies, we devise a two-layer policy scheme, one for high-level policy deciding holding or accelerating and the other for low-level policy determining the specific dwell time or increase of speed. Besides, to handle the issue that the controlling actions of agents are asynchronous and temporally extended, we establish a duration-critic module based on the Recurrent Neural Networks (RNN) mechanism to model other agents' impact during the period between two consecutive control. We evaluate the proposed framework on a simulated bus line with a quasi-real-world pattern to compare the performance of both traditional headway-based control methods and existing MARL methods. Results show that our method outperforms other baselines, not only stabilizing a strongly unstable bus line but also shortening the traveling times of passengers.