Coded Distributed Computing Under Combination Networks

被引:0
|
作者
Yang, Yongcheng [1 ]
Huang, Yifei [1 ,2 ]
Qin, Xiaohuan [3 ]
Lu, Shenglian [1 ]
机构
[1] Guangxi Normal Univ, Key Lab Educ Blockchain & Intelligent Technol, Minist Educ, Guilin 541004, Peoples R China
[2] Guilin Univ Aerosp Technol, Sch Sci, Guilin 541004, Peoples R China
[3] Guangxi Normal Univ, Sch Chem & Pharmaceut Sci, State Key Lab Chem & Mol Engn Med Resources, Guilin 541004, Peoples R China
关键词
coded computing; communication; MapReduce; combination network; MAPREDUCE;
D O I
10.3390/e27030311
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
Coded distributed computing (CDC) is a powerful approach to reduce the communication overhead in distributed computing frameworks by utilizing coding techniques. In this paper, we focus on the CDC problem in (H,L)-combination networks, where H APs act as intermediate pivots and K=HL workers are connected to different subsets of L APs. Each worker processes a subset of the input file and computes intermediate values (IVs) locally, which are then exchanged via uplink and downlink transmissions through the AP station to ensure that all workers compute their assigned output functions. In this paper, we first novelly characterize the transmission scheme for the shuffle phase from the view point of the coefficient matrix and then obtain the scheme by using the Combined Placement Delivery Array (CPDA). Compared with the baseline scheme, our scheme significantly improves the uplink and downlink communication loads while maintaining the robustness and efficiency of the combined multi-AP network.
引用
收藏
页数:12
相关论文
共 50 条
  • [31] An Incentive Mechanism for Resource Allocation in Coded Distributed Computing
    Ng, Jer Shyuan
    Lim, Wei Yang Bryan
    Xiong, Zehui
    Garg, Sahil
    Niyato, Dusit
    Leung, Cyril
    2021 IEEE 15TH INTERNATIONAL CONFERENCE ON BIG DATA SCIENCE AND ENGINEERING (BIGDATASE 2021), 2021, : 15 - 22
  • [32] How to Optimally Allocate Resources for Coded Distributed Computing?
    Yu, Qian
    Li, Songze
    Maddah-Ali, Mohammad Ali
    Avestimehr, A. Salman
    2017 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS (ICC), 2017,
  • [33] Coded Reactive Stragglers Mitigation in Distributed Computing Systems
    Ardakani, Maryam Haghighi
    Ardakani, Masoud
    Tellambura, Chintha
    ICC 2023-IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS, 2023, : 3750 - 3755
  • [34] Coded Distributed Computing over Packet Erasure Channels
    Han, Dong-Jun
    Sohn, Jy-Yong
    Moon, Jaekyun
    2019 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT), 2019, : 717 - 721
  • [35] Coded Reactive Stragglers Mitigation in Distributed Computing Systems
    Ardakani, Maryam Haghighi
    Ardakani, Masoud
    Tellambura, Chintha
    IEEE TRANSACTIONS ON COMMUNICATIONS, 2024, 72 (08) : 4527 - 4537
  • [36] Coded Distributed Computing: Fundamental Limits and Practical Challenges
    Li, Songze
    Yu, Qian
    Maddah-Ali, Mohammad Ali
    Avestimehr, A. Salman
    2016 50TH ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS AND COMPUTERS, 2016, : 509 - 513
  • [37] Opportunistic Coded Distributed Computing: An Evolutionary Game Approach
    Han, Yue
    Niyato, Dusit
    Leung, Cyril
    Kim, Dong In
    IWCMC 2021: 2021 17TH INTERNATIONAL WIRELESS COMMUNICATIONS & MOBILE COMPUTING CONFERENCE (IWCMC), 2021, : 1430 - 1435
  • [38] Coded convolution for parallel and distributed computing within a deadline
    Dutta, Sanghamitra
    Cadambe, Viveck
    Grover, Pulkit
    2017 IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT), 2017, : 2403 - 2407
  • [39] Coded Distributed Computing: Straggling Servers and Multistage Dataflows
    Li, Songze
    Maddah-Ali, Mohammad Ali
    Avestimehr, A. Salman
    2016 54TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING (ALLERTON), 2016, : 164 - 171
  • [40] A New Combinatorial Coded Design for Heterogeneous Distributed Computing
    Woolsey, Nicholas
    Chen, Rong-Rong
    Ji, Mingyue
    IEEE TRANSACTIONS ON COMMUNICATIONS, 2021, 69 (09) : 5672 - 5685