On the Optimal Recovery Threshold of Coded Matrix Multiplication

被引:123
|
作者
Dutta, Sanghamitra [1 ]
Fahim, Mohammad [2 ]
Haddadpour, Farzin [2 ]
Jeong, Haewon [1 ]
Cadambe, Viveck [2 ]
Grover, Pulkit [1 ]
机构
[1] Carnegie Mellon Univ, Dept Elect & Comp Engn, Pittsburgh, PA 15213 USA
[2] Penn State Univ, Dept Elect Engn, University Pk, PA 16802 USA
基金
美国国家科学基金会;
关键词
Coded computing; distributed computing; matrix multiplication; recovery threshold; stragglers; VANDERMONDE; INVERSION; MAPREDUCE; ALGORITHM; SYSTEMS; STORAGE;
D O I
10.1109/TIT.2019.2929328
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
We provide novel coded computation strategies for distributed matrix-matrix products that outperform the recent "Polynomial code" constructions in recovery threshold, i.e., the required number of successful workers. When a fixed 1/m fraction of each matrix can be stored at each worker node, Polynomial codes require m(2) successful workers, while our MatDot codes only require 2m - 1 successful workers. However, MatDot codes have higher computation cost per worker and higher communication cost from each worker to the fusion node. We also provide a systematic construction of MatDot codes. Furthermore, we propose "PolyDot" coding that interpolates between Polynomial codes and MatDot codes to trade off computation/communication costs and recovery thresholds. Finally, we demonstrate a novel coding technique for multiplying n matrices (n >= 3) using ideas from MatDot and PolyDot codes.
引用
收藏
页码:278 / 301
页数:24
相关论文
共 50 条
  • [1] On the Optimal Recovery Threshold of Coded Matrix Multiplication
    Fahim, Mohammad
    Jeong, Haewon
    Haddadpour, Farzin
    Dutta, Sanghamitra
    Cadambe, Viveck
    Grover, Pulkit
    [J]. 2017 55TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING (ALLERTON), 2017, : 1264 - 1270
  • [2] Matrix Sparsification for Coded Matrix Multiplication
    Suh, Geewon
    Lee, Kangwook
    Suh, Changho
    [J]. 2017 55TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING (ALLERTON), 2017, : 1271 - 1278
  • [3] Coded Sparse Matrix Multiplication
    Wang, Sinong
    Liu, Jiashang
    Shroff, Ness
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 80, 2018, 80
  • [4] Private Coded Matrix Multiplication
    Kim, Minchul
    Yang, Heecheol
    Lee, Jungwoo
    [J]. IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, 2020, 15 : 1434 - 1443
  • [5] Hierarchical coded matrix multiplication
    Kiani, Shahrzad
    Ferdinand, Nuwan
    Draper, Stark C.
    [J]. 2019 16TH CANADIAN WORKSHOP ON INFORMATION THEORY (CWIT), 2019,
  • [6] Coded Matrix Chain Multiplication
    Fan, Xiaodi
    Saldivia, Angel
    Soto, Pedro
    Li, Jun
    [J]. 2021 IEEE/ACM 29TH INTERNATIONAL SYMPOSIUM ON QUALITY OF SERVICE (IWQOS), 2021,
  • [7] Hierarchical Coded Matrix Multiplication
    Kianidehkordi, Shahrzad
    Ferdinand, Nuwan
    Draper, Stark C.
    [J]. IEEE TRANSACTIONS ON INFORMATION THEORY, 2021, 67 (02) : 726 - 754
  • [8] Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix Multiplication
    Yu, Qian
    Maddah-Ali, Mohammad Ali
    Avestimehr, A. Salman
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 30 (NIPS 2017), 2017, 30
  • [9] Locally Recoverable Coded Matrix Multiplication
    Jeong, Haewon
    Ye, Fangwei
    Grover, Pulkit
    [J]. 2018 56TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING (ALLERTON), 2018, : 715 - 722
  • [10] Variable Coded Batch Matrix Multiplication
    Tauz, Lev
    Dolecek, Lara
    [J]. 2021 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM), 2021,