A neuro-dynamic programming approach for perimeter control of two urban regions with macroscopic fundamental diagrams

Macroscopic Fundamental Diagram (MFD) model is widely used to describe urban traffic dynamic system. Based on the MFD model, perimeter control methods are developed to ensure the efficiency of the system. However, most existing perimeter control methods would suffer from two shortcomings, i.e., linearization of the MFD function, accurate calibration of MFD and travel demand. These prerequisites would undermine the performance of the system if an accurate calibration cannot be guaranteed. On the other hand, an optimization scheme of network performance without excessive knowledge of state variables but based on traffic data is preferable. In this study, an optimal feedback controller based on the neuro-dynamic that approximates the solution of the Hamilton-Jacobi-Bellman equation (HJB) is introduced. Firstly, the value function is approximated by a neural network. Then the parameters are optimized by the policy iteration method, with the objective of minimizing the cumulative error toward set-point. Furthermore, the optimal control law constrained by a saturated operator is implemented based on real-time observations recursively. The neuro-dynamic controller is tested for the two-regional MFD system. The results confirm that the neuro-dynamic controller can regulate the tra ffic states converge to the desired uncongested equilibrium.