A hierarchical agent-based approach to simulate a dynamic decision-making process of evacuees using reinforcement learning

Simulation models are an undeniable tool to help researchers and designers forecast effects of definite policies regarding pedestrian social and collective movement behaviour. Considering both the environment's details and the complexity of human behaviour in choosing paths simultaneously is the main challenge in micro-simulation pedestrian dynamics models. This paper aims to present a novel comprehensive hierarchical agent-based simulation of pedestrian evacuation from a dynamic network of the environment using reinforcement learning, which is the closest to human behaviour among the other machine learning algorithms. In the approach, agents autonomously decide through a three-layer hierarchical model, including goal, node, and cell selection layers. A multinomial logit model is used to model the process of choosing the main movement direction at each time-step. The proposed model was successfully tested to simulate the pedestrian evacuation process from the Britomart Transport Centre platforms in Auckland during an abstract destructive event. Maximum evacuation flow, total evacuation time, average evacuation time, and average evacuation flow were investigated as dependent variables through different evacuation scenarios. The results from the approach can be used by designers and managers to optimise the quality of evacuation; also, the proposed model has the potential of becoming a potent tool for constructional management if coupled with other constructional tools.