Mean Field Multi-Agent Reinforcement Learning Method for Area Traffic Signal Control

被引:1
|
作者
Zhang, Zundong [1 ]
Zhang, Wei [1 ]
Liu, Yuke [1 ]
Xiong, Gang [2 ]
机构
[1] North China Univ Technol, Sch Elect & Control Engn, Beijing 100144, Peoples R China
[2] Chinese Acad Sci, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China
关键词
traffic engineering; area traffic signal control; mean field theory; multi-agent reinforcement learning; neural network; traffic simulation;
D O I
10.3390/electronics12224686
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Reinforcement learning is an effective method for adaptive traffic signal control in urban transportation networks. As the number of training rounds increases, the optimal control strategy is learned, and the learning capabilities of deep neural networks are further enhanced, thereby avoiding the limitations of traditional signal control methods. However, when faced with the sequential decision tasks of regional signal control, it encounters issues such as the curse of dimensionality and environmental non-stationarity. To address the limitations of traditional reinforcement learning algorithms applied to multiple intersections, the mean field theory is applied. This models the traffic signal control problem at multiple intersections within a region as interactions between individual intersections and the average effects of neighboring intersections. By decomposing the Q-function through bilateral estimation between the agent and its neighbors, this method reduces the complexity of interactions between agents while preserving global interactions between the agents. A traffic signal control model based on Mean Field Multi-Agent Reinforcement Learning (MFMARL) was constructed, containing two algorithms: Mean Field Q-Network Area Traffic Signal Control (MFQ-ATSC) and Mean Field Actor-Critic Network Area Traffic Signal Control (MFAC-ATSC). The model was validated using the SUMO simulation platform. The experimental results indicate that across different metrics, such as average speed, the mean field reinforcement learning method outperforms classical signal control methods and several existing approaches.
引用
收藏
页数:13
相关论文
共 50 条
  • [1] Multi-agent Reinforcement Learning for Traffic Signal Control
    Prabuchandran, K. J.
    Kumar, Hemanth A. N.
    Bhatnagar, Shalabh
    [J]. 2014 IEEE 17TH INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC), 2014, : 2529 - 2534
  • [2] An Improved Traffic Signal Control Method Based on Multi-agent Reinforcement Learning
    Xu, Jianyou
    Zhang, Zhichao
    Zhang, Shuo
    Miao, Jiayao
    [J]. 2021 PROCEEDINGS OF THE 40TH CHINESE CONTROL CONFERENCE (CCC), 2021, : 6612 - 6616
  • [3] Multi-agent deep reinforcement learning with traffic flow for traffic signal control
    Hou, Liang
    Huang, Dailin
    Cao, Jie
    Ma, Jialin
    [J]. JOURNAL OF CONTROL AND DECISION, 2023,
  • [4] Mean Field Multi-Agent Reinforcement Learning
    Yang, Yaodong
    Luo, Rui
    Li, Minne
    Zhou, Ming
    Zhang, Weinan
    Wang, Jun
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 80, 2018, 80
  • [5] Multi-Agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach
    Kolat, Mate
    Kovari, Balint
    Becsi, Tamas
    Aradi, Szilard
    [J]. SUSTAINABILITY, 2023, 15 (04)
  • [6] Cooperative Traffic Signal Control Based on Multi-agent Reinforcement Learning
    Gao, Ruowen
    Liu, Zhihan
    Li, Jinglin
    Yuan, Quan
    [J]. BLOCKCHAIN AND TRUSTWORTHY SYSTEMS, BLOCKSYS 2019, 2020, 1156 : 787 - 793
  • [7] Causal inference multi-agent reinforcement learning for traffic signal control
    Yang, Shantian
    Yang, Bo
    Zeng, Zheng
    Kang, Zhongfeng
    [J]. INFORMATION FUSION, 2023, 94 : 243 - 256
  • [8] Hierarchical graph multi-agent reinforcement learning for traffic signal control
    Yang, Shantian
    [J]. INFORMATION SCIENCES, 2023, 634 : 55 - 72
  • [9] Dynamic traffic signal control using mean field multi-agent reinforcement learning in large scale road-networks
    Hu, Tianfeng
    Hu, Zhiqun
    Lu, Zhaoming
    Wen, Xiangming
    [J]. IET INTELLIGENT TRANSPORT SYSTEMS, 2023, 17 (09) : 1715 - 1728
  • [10] Multi-Agent Deep Reinforcement Learning for Decentralized Cooperative Traffic Signal Control
    Zhao, Yang
    Hu, Jian-Ming
    Gao, Ming-Yang
    Zhang, Zuo
    [J]. CICTP 2020: TRANSPORTATION EVOLUTION IMPACTING FUTURE MOBILITY, 2020, : 458 - 470