Fair Coflow Scheduling via Controlled Slowdown

被引:0
|
作者
De Pellegrini, Francesco [1 ]
Gupta, Vaibhav Kumar [2 ]
El Azouzi, Rachid [1 ]
Gueye, Serigne [1 ]
Richier, Cedric [1 ]
Leguay, Jeremie [3 ]
机构
[1] Avignon Univ, Lab Informat Avignon LIA, F-84029 Avignon, France
[2] LNM Inst Informat Technol LNMIIT, Jaipur 302031, Rajasthan, India
[3] Huawei Technol, Paris Res Ctr, F-92100 Boulogne Billancourt, France
关键词
Resource management; Minimization; Optimal scheduling; Data transfer; Switches; Standards; Scheduling; Coflow scheduling; data transfer; fairness; primal-dual scheduler; progress; ALGORITHM;
D O I
10.1109/TPDS.2024.3446188
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
The average coflow completion time (CCT) is the standard performance metric in coflow scheduling. However, standard CCT minimization may introduce unfairness between the data transfer phase of different computing jobs. Thus, while progress guarantees have been introduced in the literature to mitigate this fairness issue, the trade-off between fairness and efficiency of data transfer is hard to control. This paper introduces a fairness framework for coflow scheduling based on the concept of slowdown, i.e., the performance loss of a coflow compared to isolation. By controlling the slowdown it is possible to enforce a target coflow progress while minimizing the average CCT. In the proposed framework, the minimum slowdown for a batch of coflows can be determined in polynomial time. By showing the equivalence with Gaussian elimination, slowdown constraints are introduced into primal-dual iterations of the CoFair algorithm. The algorithm extends the class of the sigma-order schedulers to solve the fair coflow scheduling problem in polynomial time. It provides a 4-approximation of the average CCT w.r.t. an optimal scheduler. Extensive numerical results demonstrate that this approach can trade off average CCT for slowdown more efficiently than existing state of the art schedulers.
引用
收藏
页码:2347 / 2360
页数:14
相关论文
共 50 条
  • [1] Fair Coflow Scheduling without Prior Knowledge
    Wang, Luping
    Wang, Wei
    2018 IEEE 38TH INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS (ICDCS), 2018, : 22 - 32
  • [2] Fair and near-optimal coflow scheduling without prior knowledge of coflow size
    Li, Chenghao
    Zhang, Huyin
    Ding, Wenjia
    Zhou, Tianying
    JOURNAL OF SUPERCOMPUTING, 2021, 77 (07): : 7690 - 7717
  • [3] Fair and near-optimal coflow scheduling without prior knowledge of coflow size
    Chenghao Li
    Huyin Zhang
    Wenjia Ding
    Tianying Zhou
    The Journal of Supercomputing, 2021, 77 : 7690 - 7717
  • [4] PROSE: Multi-round fair coflow scheduling without prior knowledge
    Zhang, Fan
    Tang, Yazhe
    Li, Xun
    Hu, Chengchen
    COMPUTER COMMUNICATIONS, 2023, 209 : 163 - 177
  • [5] Coflow Deadline Scheduling via Network-Aware Optimization
    Tseng, Shih-Hao
    Tang, Ao
    2018 56TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING (ALLERTON), 2018, : 829 - 833
  • [6] Matroid coflow scheduling
    University of California, Merced, United States
    不详
    PA, United States
    不详
    不详
    CA, United States
    Leibniz Int. Proc. Informatics, LIPIcs, 1868,
  • [7] Efficient and Fair: Information-Agnostic Online Coflow Scheduling by Combining Limited Multiplexing With DRL
    Wang, Xin
    Shen, Hong
    Tian, Hui
    IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, 2023, 20 (04): : 4572 - 4584
  • [8] Efficient Coflow Scheduling with Varys
    Chowdhury, Mosharaf
    Zhong, Yuan
    Stoica, Ion
    SIGCOMM'14: PROCEEDINGS OF THE 2014 ACM CONFERENCE ON SPECIAL INTEREST GROUP ON DATA COMMUNICATION, 2014, : 443 - 454
  • [9] Fair and Timely Scheduling via Cooperative Polling
    Krasic, Charles
    Saubhasik, Mayukh
    Sinha, Anirban
    Goel, Ashvin
    EUROSYS'09: PROCEEDINGS OF THE FOURTH EUROSYS CONFERENCE, 2009, : 103 - 115
  • [10] Efficient Coflow Scheduling with Varys
    Chowdhury, Mosharaf
    Zhong, Yuan
    Stoica, Ion
    ACM SIGCOMM COMPUTER COMMUNICATION REVIEW, 2014, 44 (04) : 443 - 454