AQMDRL: Automatic Quality of Service Architecture Based on Multistep Deep Reinforcement Learning in Software-Defined Networking

被引:5
|
作者
Chen, Junyan [1 ,2 ,3 ]
Liao, Cenhuishan [1 ]
Wang, Yong [1 ]
Jin, Lei [1 ]
Lu, Xiaoye [1 ]
Xie, Xiaolan [1 ]
Yao, Rui [1 ]
机构
[1] Guilin Univ Elect Technol, Sch Comp Sci & Informat Secur, Guilin 541004, Peoples R China
[2] Guilin Univ Elect Technol, Sch Informat & Commun, Guilin 541004, Peoples R China
[3] Northeastern Univ, Sch Comp Sci & Engn, Shenyang 110819, Peoples R China
基金
中国国家自然科学基金;
关键词
software-defined networking; deep reinforcement learning; quality of service; deep deterministic policy gradient; multistep; SumTree; OPTIMIZATION; COST;
D O I
10.3390/s23010429
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Software-defined networking (SDN) has become one of the critical technologies for data center networks, as it can improve network performance from a global perspective using artificial intelligence algorithms. Due to the strong decision-making and generalization ability, deep reinforcement learning (DRL) has been used in SDN intelligent routing and scheduling mechanisms. However, traditional deep reinforcement learning algorithms present the problems of slow convergence rate and instability, resulting in poor network quality of service (QoS) for an extended period before convergence. Aiming at the above problems, we propose an automatic QoS architecture based on multistep DRL (AQMDRL) to optimize the QoS performance of SDN. AQMDRL uses a multistep approach to solve the overestimation and underestimation problems of the deep deterministic policy gradient (DDPG) algorithm. The multistep approach uses the maximum value of the n-step action currently estimated by the neural network instead of the one-step Q-value function, as it reduces the possibility of positive error generated by the Q-value function and can effectively improve convergence stability. In addition, we adapt a prioritized experience sampling based on SumTree binary trees to improve the convergence rate of the multistep DDPG algorithm. Our experiments show that the AQMDRL we proposed significantly improves the convergence performance and effectively reduces the network transmission delay of SDN over existing DRL algorithms.
引用
收藏
页数:17
相关论文
共 50 条
  • [21] Deep Reinforcement Learning for Cost-Effective Controller Placement in Software-Defined Multihop Wireless Networking
    Zahmatkesh, Afsane
    Lung, Chung-Horng
    MACHINE LEARNING FOR NETWORKING, MLN 2021, 2022, 13175 : 131 - 147
  • [22] Service as a standard [software-defined networking concept]
    Courtney, M.
    Engineering and Technology, 2012, 7 (11): : 64 - 67
  • [23] Deep learning-based data privacy protection in software-defined industrial networking
    Wu, Wenjia
    Qi, Qi
    Yu, Xiaosheng
    COMPUTERS & ELECTRICAL ENGINEERING, 2023, 106
  • [24] Examining the Quality Metrics of a Communication Network with Distributed Software-Defined Networking Architecture
    Mehmood, Khawaja Tahir
    Atiq, Shahid
    Sajjad, Intisar Ali
    Hussain, Muhammad Majid
    Basit, Malik M. Abdul
    CMES-COMPUTER MODELING IN ENGINEERING & SCIENCES, 2024, 141 (02): : 1673 - 1708
  • [25] A Software-Defined Networking Architecture for Aerial Network Optimization
    Iqbal, Hammad
    Ma, Jamie
    Stranc, Kenneth
    Palmer, Kenneth
    Benbenek, Peter
    2016 IEEE NETSOFT CONFERENCE AND WORKSHOPS (NETSOFT), 2016, : 151 - 155
  • [26] A Configurable Packet Classification Architecture for Software-Defined Networking
    Perez, K. Guerra
    Yang, X.
    Scott-Hayward, S.
    Sezer, S.
    2014 27TH IEEE INTERNATIONAL SYSTEM-ON-CHIP CONFERENCE (SOCC), 2014, : 353 - 358
  • [27] Future Access Architecture: Software-defined Access Networking
    Zheng, Ruobin
    Yang, Wenle
    Zhou, Jun
    2014 IEEE 11TH CONSUMER COMMUNICATIONS AND NETWORKING CONFERENCE (CCNC), 2014,
  • [28] Quality of Service and Congestion Control in Software-Defined Networking Using Policy-Based Routing
    Ali, Inayat
    Hong, Seungwoo
    Cheung, Taesik
    APPLIED SCIENCES-BASEL, 2024, 14 (19):
  • [29] Proposal for Cognitive Architecture for Software-defined Wireless Networking
    Kataoka, Genta
    Kaeri, Yuki
    Manabe, Yusuke
    Sugawara, Kenji
    Suganuma, Takuo
    Shiratori, Norio
    2018 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS (SMC), 2018, : 2748 - 2753
  • [30] Deep Reinforcement Learning based Smart Mitigation of DDoS Flooding in Software-Defined Networks
    Liu, Yandong
    Dong, Mianxiong
    Otat, Kaoru
    Li, Jianhua
    Wu, Jun
    2018 IEEE 23RD INTERNATIONAL WORKSHOP ON COMPUTER AIDED MODELING AND DESIGN OF COMMUNICATION LINKS AND NETWORKS (CAMAD), 2018, : 80 - 85