Fault Tolerance Oriented SFC Optimization in SDN/NFV-Enabled Cloud Environment Based on Deep Reinforcement Learning

被引:1
|
作者
Chen, Jing [1 ]
Chen, Jia [2 ,3 ]
Guo, Kuo [2 ]
Hu, Renkun [2 ]
Zou, Tao [4 ]
Zhu, Jun [4 ]
Zhang, Hongke [2 ,3 ]
Liu, Jingjing [5 ]
机构
[1] Tsinghua Univ, Dept Comp Sci & Technol, Beijing 100084, Peoples R China
[2] Beijing Jiaotong Univ, Dept Elect & Informat Engn, Beijing 100084, Peoples R China
[3] Pengcheng Lab, Shenzhen 518055, Peoples R China
[4] Zhejiang Lab, Hangzhou 311121, Zhejiang, Peoples R China
[5] China Mobile Grp Liaoning Co Ltd, Shenyang 110179, Peoples R China
关键词
Service function chain; fault-tolerant; quality of service; elastic optimization; deep reinforcement learning; RESOURCE OPTIMIZATION; NETWORK; NFV; MIGRATION; VNF; ORCHESTRATION; MANAGEMENT; PLACEMENT; EDGE;
D O I
10.1109/TCC.2024.3357061
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In software defined network/network function virtualization (SDN/NFV)-enabled cloud environment, cloud services can be implemented as service function chains (SFCs), which consist of a series of ordered virtual network functions. However, due to fluctuations of cloud traffic and without knowledge of cloud computing network configuration, designing SFC optimization approach to obtain flexible cloud services in dynamic cloud environment is a pivotal challenge. In this paper, we propose a fault tolerance oriented SFC optimization approach based on deep reinforcement learning. We model fault tolerance oriented SFC elastic optimization problem as a Markov decision process, in which the reward is modeled as a weighted function, including minimizing energy consumption and migration cost, maximizing revenue benefit and load balancing. Then, taking binary integer programming model as constraints of quality of cloud services, we design optimization approaches for single-agent double deep Q-network (SADDQN) and multi-agent DDQN (MADDQN). Among them, MADDQN decentralizes training tasks from control plane to data plane to reduce the probability of single point of failure for the centralized controller. Experimental results show that the designed approaches have better performance. MADDQN can almost reach the upper bound of theoretical solution obtained by assuming a prior knowledge of the dynamics of cloud traffic.
引用
收藏
页码:200 / 218
页数:19
相关论文
共 50 条
  • [21] A SDN Routing Optimization Mechanism Based on Deep Reinforcement Learning
    Lan J.
    Yu C.
    Hu Y.
    Li Z.
    Dianzi Yu Xinxi Xuebao/Journal of Electronics and Information Technology, 2019, 41 (11): : 2669 - 2674
  • [22] A SDN Routing Optimization Mechanism Based on Deep Reinforcement Learning
    Lan Julong
    Yu Changhe
    Hu Yuxiang
    Li Ziyong
    JOURNAL OF ELECTRONICS & INFORMATION TECHNOLOGY, 2019, 41 (11) : 2669 - 2674
  • [23] QoS Routing Optimization Based on Deep Reinforcement Learning in SDN
    Song, Yu
    Qian, Xusheng
    Zhang, Nan
    Wang, Wei
    Xiong, Ao
    CMC-COMPUTERS MATERIALS & CONTINUA, 2024, 79 (02): : 3007 - 3021
  • [24] Fault tolerant trust based task scheduler using Harris Hawks optimization and deep reinforcement learning in multi cloud environment
    Sudheer Mangalampalli
    Ganesh Reddy Karri
    Sachi Nandan Mohanty
    Shahid Ali
    M. Ijaz Khan
    Dilsora Abduvalieva
    Fuad A. Awwad
    Emad A. A. Ismail
    Scientific Reports, 13
  • [25] Fault tolerant trust based task scheduler using Harris Hawks optimization and deep reinforcement learning in multi cloud environment
    Mangalampalli, Sudheer
    Karri, Ganesh Reddy
    Mohanty, Sachi Nandan
    Ali, Shahid
    Khan, M. Ijaz
    Abduvalieva, Dilsora
    Awwad, Fuad A.
    Ismail, Emad A. A.
    SCIENTIFIC REPORTS, 2023, 13 (01)
  • [26] Deep reinforcement learning for fault-tolerant workflow scheduling in cloud environment
    Dong, Tingting
    Xue, Fei
    Tang, Hengliang
    Xiao, Chuangbai
    APPLIED INTELLIGENCE, 2023, 53 (09) : 9916 - 9932
  • [27] Deep reinforcement learning for fault-tolerant workflow scheduling in cloud environment
    Tingting Dong
    Fei Xue
    Hengliang Tang
    Chuangbai Xiao
    Applied Intelligence, 2023, 53 : 9916 - 9932
  • [28] Handling Efficient VNF Placement with Graph-Based Reinforcement Learning for SFC Fault Tolerance
    Ros, Seyha
    Tam, Prohim
    Song, Inseok
    Kang, Seungwoo
    Kim, Seokhoon
    ELECTRONICS, 2024, 13 (13)
  • [29] Workflow scheduling based on deep reinforcement learning in the cloud environment
    Tingting Dong
    Fei Xue
    Chuangbai Xiao
    Jiangjiang Zhang
    Journal of Ambient Intelligence and Humanized Computing, 2021, 12 : 10823 - 10835
  • [30] Workflow scheduling based on deep reinforcement learning in the cloud environment
    Dong, Tingting
    Xue, Fei
    Xiao, Chuangbai
    Zhang, Jiangjiang
    JOURNAL OF AMBIENT INTELLIGENCE AND HUMANIZED COMPUTING, 2021, 12 (12) : 10823 - 10835