Yard Crane Scheduling Method Based on Deep Reinforcement Learning for the Automated Container Terminal

As the core working machinery of automated terminal yard, the dispatching of yard crane is the key to improve the efficiency of container operation. In order to minimize the waiting time of AGVs and external container trucks, a mathematical programming model for the yard crane scheduling problem is established considering complex spatio-temporal coupling characteristics and high dynamic, and a novel deep reinforcement learning method is proposed to solve the problem. The algorithm describes the yard environment close to reality through the agent definition, and improves the ability of extracting hidden state patterns through pointer network, attention mechanism and A-C architecture in the interaction design between the agent and the environment. Experiments are carried out on examples of different scales based on the actual data of Yangshan Phase IV Automated Terminal. The results show that the proposed algorithm can provide an approximately optimal crane scheduling scheme in a relatively short time, and the performance of it is about 17% better compared with state-of-art heuristic rule algorithms. Therefore, the proposed scheduling method is effective and superior, and it can provide dynamic decision support for yard operation in practice. © 2024 Chinese Mechanical Engineering Society. All rights reserved.