Scheduling divisible MapReduce computations

被引：50
|
作者：
Berlinska, J. [2 ]
Drozdowski, M. [1 ]
机构：
[1] Poznan Univ Tech, Inst Comp Sci, PL-60965 Poznan, Poland
[2] Adam Mickiewicz Univ, Fac Math & Comp Sci, PL-61614 Poznan, Poland
来源：
JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING | 2011年 / 71卷 / 03期
关键词：
Parallel processing; MapReduce; Scheduling; Divisible loads; Performance evaluation; DISTRIBUTED COMPUTATION; TREE NETWORKS; LOADS;
D O I：
10.1016/j.jpdc.2010.12.004
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
In this paper we analyze MapReduce distributed computations as a divisible load scheduling problem. The two operations of mapping and reducing can be understood as two divisible applications with precedence constraints. A divisible load model of the computation, and two load partitioning algorithms are proposed. Performance limits of MapReduce computations are investigated. To our best knowledge this is the first time that processing applications with precedence constraints have been considered on the grounds of divisible load theory. (C) 2010 Elsevier Inc. All rights reserved.
引用
收藏
页码：450 / 459
页数：10
相关论文
共 50 条
  • [1] SCHEDULING MULTILAYER DIVISIBLE COMPUTATIONS
    Berlinska, Joanna
    Drozdowski, Maciej
    RAIRO-OPERATIONS RESEARCH, 2015, 49 (02) : 339 - 368
  • [2] Scheduling Divisible Reduce Tasks in MapReduce
    Gu, Tao
    Zuo, Chuang
    Chen, Zheng
    Yang, Yulu
    Li, Tao
    2014 5TH IEEE INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING AND SERVICE SCIENCE (ICSESS), 2014, : 190 - 194
  • [3] Scheduling Divisible Loads from Multiple Input Sources in MapReduce
    Gu, Tao
    Liao, Qun
    Yang, Yulu
    Li, Tao
    2013 IEEE 16TH INTERNATIONAL CONFERENCE ON COMPUTATIONAL SCIENCE AND ENGINEERING (CSE 2013), 2013, : 1263 - 1270
  • [4] An Efficiency-Aware Scheduling for Data-Intensive Computations on MapReduce Clusters
    Zhao, Hui
    Yang, Shuqiang
    Fan, Hua
    Chen, Zhikun
    Xu, Jinghu
    IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2013, E96D (12): : 2654 - 2662
  • [5] Isoefficiency Maps for Divisible Computations
    Drozdowski, Maciej
    Wielebski, Lukasz
    IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2010, 21 (06) : 872 - 880
  • [6] Scheduling multiple divisible loads
    Drozdowski, M
    Lawenda, M
    Guinand, F
    INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS, 2006, 20 (01): : 19 - 30
  • [7] Scheduling divisible jobs on hypercubes
    Blazewicz, J
    Drozdowski, M
    PARALLEL COMPUTING, 1995, 21 (12) : 1945 - 1956
  • [8] Connecting MapReduce Computations to Realistic Machine Models
    Sanders, Peter
    2020 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2020, : 84 - 93
  • [9] A Cost-Efficient Resource Provisioning and Scheduling Approach for Deadline-Sensitive MapReduce Computations in Cloud Environment
    Jabbari, Amir
    Masoumiyan, Farzaneh
    Hu, Shuwen
    Tang, Maolin
    Tian, Yu-Chu
    2021 IEEE 14TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING (CLOUD 2021), 2021, : 600 - 608
  • [10] MapReduce scheduling algorithms: a review
    Hashem, Ibrahim Abaker Targio
    Anuar, Nor Badrul
    Marjani, Mohsen
    Ahmed, Ejaz
    Chiroma, Haruna
    Firdaus, Ahmad
    Abdullah, Muhamad Taufik
    Alotaibi, Faiz
    Ali, Waleed Kamaleldin Mahmoud
    Yaqoob, Ibrar
    Gani, Abdullah
    JOURNAL OF SUPERCOMPUTING, 2020, 76 (07): : 4915 - 4945
12345