AQMDRL: Automatic Quality of Service Architecture Based on Multistep Deep Reinforcement Learning in Software-Defined Networking

被引:5
|
作者
Chen, Junyan [1 ,2 ,3 ]
Liao, Cenhuishan [1 ]
Wang, Yong [1 ]
Jin, Lei [1 ]
Lu, Xiaoye [1 ]
Xie, Xiaolan [1 ]
Yao, Rui [1 ]
机构
[1] Guilin Univ Elect Technol, Sch Comp Sci & Informat Secur, Guilin 541004, Peoples R China
[2] Guilin Univ Elect Technol, Sch Informat & Commun, Guilin 541004, Peoples R China
[3] Northeastern Univ, Sch Comp Sci & Engn, Shenyang 110819, Peoples R China
基金
中国国家自然科学基金;
关键词
software-defined networking; deep reinforcement learning; quality of service; deep deterministic policy gradient; multistep; SumTree; OPTIMIZATION; COST;
D O I
10.3390/s23010429
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Software-defined networking (SDN) has become one of the critical technologies for data center networks, as it can improve network performance from a global perspective using artificial intelligence algorithms. Due to the strong decision-making and generalization ability, deep reinforcement learning (DRL) has been used in SDN intelligent routing and scheduling mechanisms. However, traditional deep reinforcement learning algorithms present the problems of slow convergence rate and instability, resulting in poor network quality of service (QoS) for an extended period before convergence. Aiming at the above problems, we propose an automatic QoS architecture based on multistep DRL (AQMDRL) to optimize the QoS performance of SDN. AQMDRL uses a multistep approach to solve the overestimation and underestimation problems of the deep deterministic policy gradient (DDPG) algorithm. The multistep approach uses the maximum value of the n-step action currently estimated by the neural network instead of the one-step Q-value function, as it reduces the possibility of positive error generated by the Q-value function and can effectively improve convergence stability. In addition, we adapt a prioritized experience sampling based on SumTree binary trees to improve the convergence rate of the multistep DDPG algorithm. Our experiments show that the AQMDRL we proposed significantly improves the convergence performance and effectively reduces the network transmission delay of SDN over existing DRL algorithms.
引用
收藏
页数:17
相关论文
共 50 条
  • [41] Token as a Service for Software-Defined Zero Trust Networking
    Erel-Ozcevik, Muge
    JOURNAL OF NETWORK AND SYSTEMS MANAGEMENT, 2025, 33 (01)
  • [42] Video Streaming Service Identification on Software-Defined Networking
    Castaneda Herrera, Luis Miguel
    Campo Munoz, Wilmar Yesid
    Duque-Torres, Alejandra
    INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL, 2021, 16 (05)
  • [43] A software-defined radio testbed for deep learning-based automatic modulation classification
    Ponnaluru, Sowjanya
    Penke, Satyanarayana
    INTERNATIONAL JOURNAL OF COMMUNICATION SYSTEMS, 2020, 33 (15)
  • [44] A Novel Traffic Classification Approach by Employing Deep Learning on Software-Defined Networking
    Nunez-Agurto, Daniel
    Fuertes, Walter
    Marrone, Luis
    Benavides-Astudillo, Eduardo
    Coronel-Guerrero, Christian
    Perez, Franklin
    FUTURE INTERNET, 2024, 16 (05)
  • [45] Managing Energy Consumption of Devices with Multiconnectivity by Deep Learning and Software-Defined Networking
    Shams, Ramiza
    Abdrabou, Atef
    Al Bataineh, Mohammad
    Noordin, Kamarul Ariffin
    SENSORS, 2023, 23 (18)
  • [46] Deep learning and software-defined networks: Towards secure IoT architecture
    Dawoud, Ahmed
    Shahristani, Seyed
    Raun, Chun
    INTERNET OF THINGS, 2018, 3-4 : 82 - 89
  • [47] Q-placement: Reinforcement-Learning-Based Service Placement in Software-Defined Networks
    Zhang, Ziyao
    Ma, Liang
    Leung, Kin K.
    Tassiulas, Leandros
    Tucker, Jeremy
    2018 IEEE 38TH INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS (ICDCS), 2018, : 1527 - 1532
  • [48] Machine learning based malicious payload identification in software-defined networking
    Cheng, Qiumei
    Wu, Chunming
    Zhou, Haifeng
    Kong, Dezhang
    Zhang, Dong
    Xing, Junchi
    Ruan, Wei
    JOURNAL OF NETWORK AND COMPUTER APPLICATIONS, 2021, 192
  • [49] Software-Defined Networking (SDN) based VANET Architecture: Mitigation of Traffic Congestion
    Adbeb, Tesfanesh
    Di, Wu
    Ibrar, Muhammad
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2020, 11 (03) : 706 - 714
  • [50] Machine Learning based Software-Defined Networking Traffic Classification System
    Vulpe, Alexandru
    Girla, Ionut
    Craciunescu, Razvan
    Berceanu, Madalina Georgiana
    2021 IEEE INTERNATIONAL BLACK SEA CONFERENCE ON COMMUNICATIONS AND NETWORKING (IEEE BLACKSEACOM), 2021, : 377 - 381