Deep Reinforcement Learning-Based Multi-Agent Algorithm for Vehicle Routing Problem in Complex Logistics Scenarios

The Vehicle Routing Problem with Simultaneous Pickup-Delivery and Time Windows (VRPSPDTW) is a highly challenging issue in complex logistics distribution scenarios, requiring an optimal balance between cost and efficiency. Traditional methods often rely on single heuristic or metaheuristic algorithms, which perform not so well when dealing with VRPSPDTW. To overcome this challenge, we propose a deep reinforcement learning-based multi-agent algorithm (DRL-MA) to tackle the VRPSPDTW. Our algorithm includes explorative, exploitative, and perturbative agents, which are responsible for balancing exploration and exploitation. The action space of each agent comprises a combination of neighborhood operators, and then the Deep Q-network (DQN) is used to learn effective neighborhood transition sequences from a long-term perspective, which can effectively explore large and complex solution spaces. The cooperation and competition among agents during the search process offer a more flexible and effective strategy. Experimental studies conducted on a real test suite of large-scale VRPSPDTW instances validate the superiority of our proposed DRL-MA over some state-of-the-art algorithms.