AQMDRL: Automatic Quality of Service Architecture Based on Multistep Deep Reinforcement Learning in Software-Defined Networking

被引:5
|
作者
Chen, Junyan [1 ,2 ,3 ]
Liao, Cenhuishan [1 ]
Wang, Yong [1 ]
Jin, Lei [1 ]
Lu, Xiaoye [1 ]
Xie, Xiaolan [1 ]
Yao, Rui [1 ]
机构
[1] Guilin Univ Elect Technol, Sch Comp Sci & Informat Secur, Guilin 541004, Peoples R China
[2] Guilin Univ Elect Technol, Sch Informat & Commun, Guilin 541004, Peoples R China
[3] Northeastern Univ, Sch Comp Sci & Engn, Shenyang 110819, Peoples R China
基金
中国国家自然科学基金;
关键词
software-defined networking; deep reinforcement learning; quality of service; deep deterministic policy gradient; multistep; SumTree; OPTIMIZATION; COST;
D O I
10.3390/s23010429
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Software-defined networking (SDN) has become one of the critical technologies for data center networks, as it can improve network performance from a global perspective using artificial intelligence algorithms. Due to the strong decision-making and generalization ability, deep reinforcement learning (DRL) has been used in SDN intelligent routing and scheduling mechanisms. However, traditional deep reinforcement learning algorithms present the problems of slow convergence rate and instability, resulting in poor network quality of service (QoS) for an extended period before convergence. Aiming at the above problems, we propose an automatic QoS architecture based on multistep DRL (AQMDRL) to optimize the QoS performance of SDN. AQMDRL uses a multistep approach to solve the overestimation and underestimation problems of the deep deterministic policy gradient (DDPG) algorithm. The multistep approach uses the maximum value of the n-step action currently estimated by the neural network instead of the one-step Q-value function, as it reduces the possibility of positive error generated by the Q-value function and can effectively improve convergence stability. In addition, we adapt a prioritized experience sampling based on SumTree binary trees to improve the convergence rate of the multistep DDPG algorithm. Our experiments show that the AQMDRL we proposed significantly improves the convergence performance and effectively reduces the network transmission delay of SDN over existing DRL algorithms.
引用
收藏
页数:17
相关论文
共 50 条
  • [1] ALBRL: Automatic Load-Balancing Architecture Based on Reinforcement Learning in Software-Defined Networking
    Chen, Junyan
    Wang, Yong
    Ou, Jiangtao
    Fan, Chengyuan
    Lu, Xiaoye
    Liao, Cenhuishan
    Huang, Xuefeng
    Zhang, Hongmei
    WIRELESS COMMUNICATIONS & MOBILE COMPUTING, 2022, 2022
  • [2] Software-defined networking QoS optimization based on deep reinforcement learning
    Lan J.
    Zhang X.
    Hu Y.
    Sun P.
    Tongxin Xuebao/Journal on Communications, 2019, 40 (12): : 60 - 67
  • [3] DRSIR: A Deep Reinforcement Learning Approach for Routing in Software-Defined Networking
    Casas-Velasco, Daniela M.
    Rendon, Oscar Mauricio Caicedo
    da Fonseca, Nelson L. S.
    IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, 2022, 19 (04): : 4807 - 4820
  • [4] Intelligent Routing Based on Reinforcement Learning for Software-Defined Networking
    Casas-Velasco, Daniela M.
    Rendon, Oscar Mauricio Caicedo
    da Fonseca, Nelson L. S.
    IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, 2021, 18 (01): : 870 - 881
  • [5] Reinforcement Learning for Autonomous Defence in Software-Defined Networking
    Han, Yi
    Rubinstein, Benjamin I. P.
    Abraham, Tamas
    Alpcan, Tansu
    De Vel, Olivier
    Erfani, Sarah
    Hubczenko, David
    Leckie, Christopher
    Montague, Paul
    DECISION AND GAME THEORY FOR SECURITY, GAMESEC 2018, 2018, 11199 : 145 - 165
  • [6] An algorithm to improve quality of service for software-defined networking
    Abdelghany H.M.
    Zaki F.W.
    Ashour M.M.
    Journal of Ambient Intelligence and Humanized Computing, 2023, 14 (08) : 10823 - 10832
  • [7] Adaptive Configuration with Deep Reinforcement Learning in Software-Defined Time-Sensitive Networking
    Guo, Mengjie
    Shou, Guochu
    Liu, Yaqiong
    Hu, Yihong
    PROCEEDINGS OF 2024 IEEE/IFIP NETWORK OPERATIONS AND MANAGEMENT SYMPOSIUM, NOMS 2024, 2024,
  • [8] Deep Learning Based Anomaly Detection Scheme in Software-Defined Networking
    Qin, Yang
    Wei, Junjie
    Yang, Weihong
    2019 20TH ASIA-PACIFIC NETWORK OPERATIONS AND MANAGEMENT SYMPOSIUM (APNOMS), 2019,
  • [9] Deep Reinforcement Learning-Based Routing on Software-Defined Networks
    Kim, Gyungmin
    Kim, Yohan
    Lim, Hyuk
    IEEE ACCESS, 2022, 10 : 18121 - 18133
  • [10] Intelligent Quality of Service Routing in Software-Defined Satellite Networking
    Wu, Shuai
    Yang, Lei
    Guo, Jianming
    Chen, Quan
    Liu, Xianfeng
    Fan, Chengguang
    IEEE ACCESS, 2019, 7 : 155281 - 155298