A Safe Training Approach for Deep Reinforcement Learning-based Traffic Engineering

被引:0
|
作者
Wang, Linghao [1 ,2 ]
Wang, Miao [1 ]
Zhang, Yujun [1 ,2 ,3 ]
机构
[1] Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China
[2] Univ Chinese Acad Sci, Beijing, Peoples R China
[3] Nanjing Inst Informat Superbahn, Nanjing, Peoples R China
基金
美国国家科学基金会;
关键词
Traffic Engineering; Safe Reinforcement Learning; Learning from Demonstration;
D O I
10.1109/ICC45855.2022.9838944
中图分类号
TN [电子技术、通信技术];
学科分类号
0809 ;
摘要
Traffic engineering (TE) is fundamental and important in modern communication networks. Deep reinforcement learning (DRL)-based TE solutions can solve TE in a data-driven and model-free way thus have attracted much attention recently. However, most of these solutions ignore that TE is a real-world application and there are challenges applying DRL to real-world TE like: (1) Efficiency. Existing learning-from-scratch DRL agent needs long-time interactions to find solutions better than traditional methods. (2) Safety. Existing DRL-based solutions make TE decisions without considering safety constraints, poor decisions may be made and cause significant performance degradation. In this paper, we propose a safe training approach for DRL-based TE, which tries to address the above two problems. It focuses on making full use of data and ensuring safety so that DRL agent for TE can learn more quickly and possibly poor decisions will not be applied to real environment. We implemented the proposed method in ns-3 and simulation results show that our method performs better with faster convergence rate compared to other DRL-based methods while ensuring the safety of the performed TE decisions.
引用
收藏
页码:1450 / 1455
页数:6
相关论文
共 50 条
  • [1] Deep Reinforcement Learning-based Traffic Signal Control
    Ruan, Junyun
    Tang, Jinzhuo
    Gao, Ge
    Shi, Tianyu
    Khamis, Alaa
    2023 IEEE INTERNATIONAL CONFERENCE ON SMART MOBILITY, SM, 2023, : 21 - 26
  • [2] A Scalable Deep Reinforcement Learning Approach for Traffic Engineering Based on Link Control
    Sun, Penghao
    Lan, Julong
    Li, Junfei
    Zhang, Jianpeng
    Hu, Yuxiang
    Guo, Zehua
    IEEE COMMUNICATIONS LETTERS, 2021, 25 (01) : 171 - 175
  • [3] Transfer Learning-Based Deep Reinforcement Learning Approach for Robust Route Guidance in Mixed Traffic Environment
    Lee, Donghoun
    IEEE ACCESS, 2024, 12 : 61667 - 61680
  • [4] A Deep Reinforcement Learning-Based Approach in Porker Game
    Kong, Yan
    Rui, Yefeng
    Hsia, Chih-Hsien
    Journal of Computers (Taiwan), 2023, 34 (02) : 41 - 51
  • [5] Computing on Wheels: A Deep Reinforcement Learning-Based Approach
    Kazmi, S. M. Ahsan
    Tai Manh Ho
    Tuong Tri Nguyen
    Fahim, Muhammad
    Khan, Adil
    Piran, Md Jalil
    Baye, Gaspard
    IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2022, 23 (11) : 22535 - 22548
  • [6] A deep reinforcement learning-based cooperative approach formulti-intersection traffic signal control
    Haddad, Tarek Amine
    Hedjazi, Djalal
    Aouag, Sofiane
    ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2022, 114
  • [7] A deep reinforcement learning-based cooperative approach for multi-intersection traffic signal control
    Haddad, Tarek Amine
    Hedjazi, Djalal
    Aouag, Sofiane
    Engineering Applications of Artificial Intelligence, 2022, 114
  • [8] Safe and Accelerated Deep Reinforcement Learning-Based O-RAN Slicing: A Hybrid Transfer Learning Approach
    Nagib, Ahmad M.
    Abou-Zeid, Hatem
    Hassanein, Hossam S.
    IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, 2024, 42 (02) : 310 - 325
  • [9] TITE: A transformer-based deep reinforcement learning approach for traffic engineering in hybrid SDN with dynamic traffic
    Lin, Bin
    Guo, Yingya
    Luo, Huan
    Ding, Mingjie
    FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2024, 161 : 95 - 105
  • [10] A deep reinforcement learning-based approach for the residential appliances scheduling
    Li, Sichen
    Cao, Di
    Huang, Qi
    Zhang, Zhenyuan
    Chen, Zhe
    Blaabjerg, Frede
    Hu, Weihao
    ENERGY REPORTS, 2022, 8 : 1034 - 1042