AQMDRL: Automatic Quality of Service Architecture Based on Multistep Deep Reinforcement Learning in Software-Defined Networking

被引:5
|
作者
Chen, Junyan [1 ,2 ,3 ]
Liao, Cenhuishan [1 ]
Wang, Yong [1 ]
Jin, Lei [1 ]
Lu, Xiaoye [1 ]
Xie, Xiaolan [1 ]
Yao, Rui [1 ]
机构
[1] Guilin Univ Elect Technol, Sch Comp Sci & Informat Secur, Guilin 541004, Peoples R China
[2] Guilin Univ Elect Technol, Sch Informat & Commun, Guilin 541004, Peoples R China
[3] Northeastern Univ, Sch Comp Sci & Engn, Shenyang 110819, Peoples R China
基金
中国国家自然科学基金;
关键词
software-defined networking; deep reinforcement learning; quality of service; deep deterministic policy gradient; multistep; SumTree; OPTIMIZATION; COST;
D O I
10.3390/s23010429
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Software-defined networking (SDN) has become one of the critical technologies for data center networks, as it can improve network performance from a global perspective using artificial intelligence algorithms. Due to the strong decision-making and generalization ability, deep reinforcement learning (DRL) has been used in SDN intelligent routing and scheduling mechanisms. However, traditional deep reinforcement learning algorithms present the problems of slow convergence rate and instability, resulting in poor network quality of service (QoS) for an extended period before convergence. Aiming at the above problems, we propose an automatic QoS architecture based on multistep DRL (AQMDRL) to optimize the QoS performance of SDN. AQMDRL uses a multistep approach to solve the overestimation and underestimation problems of the deep deterministic policy gradient (DDPG) algorithm. The multistep approach uses the maximum value of the n-step action currently estimated by the neural network instead of the one-step Q-value function, as it reduces the possibility of positive error generated by the Q-value function and can effectively improve convergence stability. In addition, we adapt a prioritized experience sampling based on SumTree binary trees to improve the convergence rate of the multistep DDPG algorithm. Our experiments show that the AQMDRL we proposed significantly improves the convergence performance and effectively reduces the network transmission delay of SDN over existing DRL algorithms.
引用
收藏
页数:17
相关论文
共 50 条
  • [31] Deep Reinforcement Learning Based Method for the Rule Placement Problem in Software-Defined Networks
    Jimenez-Lazaro, Manuel
    Berrocal, Javier
    Galan-Jimenez, Jaime
    PROCEEDINGS OF THE IEEE/IFIP NETWORK OPERATIONS AND MANAGEMENT SYMPOSIUM 2022, 2022,
  • [32] Software-Defined Architecture for Flying Ubiquitous Sensor Networking
    Kirichek, Ruslan
    Vladyko, Andrei
    Paramonov, Alexander
    Koucheryavy, Andrey
    2017 19TH INTERNATIONAL CONFERENCE ON ADVANCED COMMUNICATIONS TECHNOLOGY (ICACT) - OPENING NEW ERA OF SMART SOCIETY, 2017, : 158 - 162
  • [33] Deep Reinforcement Learning for Multimedia Traffic Control in Software Defined Networking
    Huang, Xiaohong
    Yuan, Tingting
    Qiao, Guanhua
    Ren, Yizhi
    IEEE NETWORK, 2018, 32 (06): : 35 - 41
  • [34] Deep Reinforcement Learning for the management of Software-Defined Networks in Smart Farming
    Alonso, Ricardo S.
    Sitton-Candanedo, Ines
    Casado-Vara, Roberto
    Prieto, Javier
    Corchado, Juan M.
    2020 INTERNATIONAL CONFERENCE ON OMNI-LAYER INTELLIGENT SYSTEMS (IEEE COINS 2020), 2020, : 135 - 140
  • [35] Introducing a Test Framework for Quality of Service Mechanisms in the Context of Software-Defined Networking
    Regencia, Josiah Eleazar T.
    Yu, William Emmanuel S.
    PROCEEDINGS OF SIXTH INTERNATIONAL CONGRESS ON INFORMATION AND COMMUNICATION TECHNOLOGY (ICICT 2021), VOL 2, 2022, 236 : 687 - 699
  • [36] DROM: Optimizing the Routing in Software-Defined Networks With Deep Reinforcement Learning
    Yu, Changhe
    Lan, Julong
    Guo, Zehua
    Hu, Yuxiang
    IEEE ACCESS, 2018, 6 : 64533 - 64539
  • [37] A new attacks intrusion detection model based on deep learning in Software-Defined Networking Environments
    Yang, Jikun
    2024 4TH INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND INTELLIGENT SYSTEMS ENGINEERING, MLISE 2024, 2024, : 430 - 436
  • [38] StEERING: A Software-Defined Networking for Inline Service Chaining
    Zhang, Ying
    Beheshti, Neda
    Beliveau, Ludovic
    Lefebvre, Geoffrey
    Manghirmalani, Ravi
    Mishra, Ramesh
    Patney, Ritun
    Shirazipour, Meral
    Subrahmaniam, Ramesh
    Truchan, Catherine
    Tatipamula, Mallik
    2013 21ST IEEE INTERNATIONAL CONFERENCE ON NETWORK PROTOCOLS (ICNP), 2013,
  • [39] Deep Learning-Based Approach for Detecting DDoS Attack on Software-Defined Networking Controller
    Mansoor, Amran
    Anbar, Mohammed
    Bahashwan, Abdullah Ahmed
    Alabsi, Basim Ahmad
    Rihan, Shaza Dawood Ahmed
    SYSTEMS, 2023, 11 (06):
  • [40] Deep Learning-Based Traffic Prediction for Energy Efficiency Optimization in Software-Defined Networking
    Chen, Xiangyi
    Wang, Xingwei
    Yi, Bo
    He, Qiang
    Huang, Min
    IEEE SYSTEMS JOURNAL, 2021, 15 (04): : 5583 - 5594