Digital-Twin-Assisted Resource Allocation for Network Slicing in Industry 4.0 and Beyond Using Distributed Deep Reinforcement Learning

被引:6
|
作者
Tang, Lun [1 ,2 ]
Du, Yucong [1 ,2 ]
Liu, Qinghai [1 ,2 ]
Li, Jinyu [1 ,2 ]
Li, Shirui [1 ,2 ]
Chen, Qianbin [1 ,2 ]
机构
[1] Chongqing Univ Posts & Telecommun, Sch Commun & Informat Engn, Chongqing 400065, Peoples R China
[2] Chongqing Univ Posts & Telecommun, Chongqing Key Lab Mobile Commun, Chongqing 400065, Peoples R China
基金
中国国家自然科学基金;
关键词
Digital twin network (DTN); Industry 4.0 and Beyond; Industry Internet of Things; prioritized experience replay (PER); resource allocation; service equilibrium; TRANSPORTATION;
D O I
10.1109/JIOT.2023.3274163
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Personalization is one of the primary emerging trends in Industry 4.0 and Beyond. Highly personalized services will present a significant challenge to the existing algorithms for network slicing (NS) and resource allocation, leading to issues, such as nonequilibratory resource allocation, in which some services are sacrificed for the maximum total reward of the algorithm, excessive cost, and slow algorithm convergence. A digital twin network (DTN) is offered as a novel solution to the challenges listed above. By integrating the DTN and IIoT NS, we propose a DTN-assisted industry Internet of Things NS (DTN-IIoT NS) architecture for personalized IIoT services in Industry 4.0 and Beyond. The DTN-IIoT NS architecture consists of three layers, three modules, and two closed loops. On the basis of the aforementioned architecture, we focus on the resource allocation process in DTN-IIoT NS, model the DT-assisted resource allocation for highly personalized IIoT services, propose the service equilibrium rate, and formulate the optimization problem aiming at maximizing the equilibrium rate weighted net profit of network providers. Then, we propose a dual-channel weighted (DCW) Critic network for service equilibrium in DTN-IIoT NS resource allocation and the matching Improved prioritized experience replay (PER) to enhance convergent speed. In addition, we present a distributed DT-assisted DCW-PER multiagent deep deterministic policy gradient (PER-DCW MADDPG) algorithm for the resource allocation process in DTN-IIoT NS. Simulation results indicate that the PER-DCW MADDPG algorithm can produce a better service equilibrium and accelerate the convergence speed of the algorithm.
引用
收藏
页码:16989 / 17006
页数:18
相关论文
共 50 条
  • [1] DeepSlicing: Deep Reinforcement Learning Assisted Resource Allocation for Network Slicing
    Liu, Qiang
    Han, Tao
    Zhang, Ning
    Wang, Ye
    [J]. 2020 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM), 2020,
  • [2] Dynamic Resource Allocation in Network Slicing with Deep Reinforcement Learning
    Cai, Yue
    Cheng, Peng
    Chen, Zhuo
    Xiang, Wei
    Vucetic, Branka
    Li, Yonghui
    [J]. IEEE CONFERENCE ON GLOBAL COMMUNICATIONS, GLOBECOM, 2023, : 2955 - 2960
  • [3] Deep Reinforcement Learning for Online Resource Allocation in Network Slicing
    Cai, Yue
    Cheng, Peng
    Chen, Zhuo
    Ding, Ming
    Vucetic, Branka
    Li, Yonghui
    [J]. IEEE TRANSACTIONS ON MOBILE COMPUTING, 2024, 23 (06) : 7099 - 7116
  • [4] Radio Resource Allocation Method for Network Slicing using Deep Reinforcement Learning
    Abiko, Yu
    Saito, Takato
    Ikeda, Daizo
    Ohta, Ken
    Mizuno, Tadanori
    Mineno, Hiroshi
    [J]. 2020 34TH INTERNATIONAL CONFERENCE ON INFORMATION NETWORKING (ICOIN 2020), 2020, : 420 - 425
  • [5] A Digital-Twin-Assisted Fault Diagnosis Using Deep Transfer Learning
    Xu, Yan
    Sun, Yanming
    Liu, Xiaolong
    Zheng, Yonghua
    [J]. IEEE ACCESS, 2019, 7 : 19990 - 19999
  • [6] Deep Reinforcement Learning for Resource Allocation with Network Slicing in Cognitive Radio Network*
    Yuan, Siyu
    Zhang, Yong
    Qie, Wenbo
    Ma, Tengteng
    Li, Sisi
    [J]. COMPUTER SCIENCE AND INFORMATION SYSTEMS, 2021, 18 (03) : 979 - 999
  • [7] Distributed Deep Reinforcement Learning Resource Allocation Scheme For Industry 4.0 Device-To-Device Scenarios
    Burgueno, Jesus
    Adeogun, Ramoni
    Bruun, Rasmus Liborius
    Garcia, C. Santiago Morejon
    de-la-Bandera, Isabel
    Barco, Raquel
    [J]. 2021 IEEE 94TH VEHICULAR TECHNOLOGY CONFERENCE (VTC2021-FALL), 2021,
  • [8] Deep Reinforcement Learning Based Resource Allocation for Network Slicing With Massive MIMO
    Yan, Dandan
    Ng, Benjamin K.
    Ke, Wei
    Lam, Chan-Tong
    [J]. IEEE ACCESS, 2023, 11 : 75899 - 75911
  • [9] Constrained Reinforcement Learning for Resource Allocation in Network Slicing
    Xu, Yizhen
    Zhao, Zhengyang
    Cheng, Peng
    Chen, Zhuo
    Ding, Ming
    Vucetic, Branka
    Li, Yonghui
    [J]. IEEE COMMUNICATIONS LETTERS, 2021, 25 (05) : 1554 - 1558
  • [10] Resource Allocation Method for Network Slicing Using Constrained Reinforcement Learning
    Liu, Yongshuai
    Ding, Jiaxin
    Liu, Xin
    [J]. 2021 IFIP NETWORKING CONFERENCE AND WORKSHOPS (IFIP NETWORKING), 2021,