Scheduling divisible MapReduce computations

被引：50
|
作者：
Berlinska, J. [2 ]
Drozdowski, M. [1 ]
机构：
[1] Poznan Univ Tech, Inst Comp Sci, PL-60965 Poznan, Poland
[2] Adam Mickiewicz Univ, Fac Math & Comp Sci, PL-61614 Poznan, Poland
来源：
JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING | 2011年 / 71卷 / 03期
关键词：
Parallel processing; MapReduce; Scheduling; Divisible loads; Performance evaluation; DISTRIBUTED COMPUTATION; TREE NETWORKS; LOADS;
D O I：
10.1016/j.jpdc.2010.12.004
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
In this paper we analyze MapReduce distributed computations as a divisible load scheduling problem. The two operations of mapping and reducing can be understood as two divisible applications with precedence constraints. A divisible load model of the computation, and two load partitioning algorithms are proposed. Performance limits of MapReduce computations are investigated. To our best knowledge this is the first time that processing applications with precedence constraints have been considered on the grounds of divisible load theory. (C) 2010 Elsevier Inc. All rights reserved.
引用
收藏
页码：450 / 459
页数：10
相关论文
共 50 条
  • [31] Task scheduling for MapReduce in heterogeneous networks
    Wang, Jia
    Li, Xiaoping
    CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2016, 19 (01): : 197 - 210
  • [32] Adaptive MapReduce Scheduling in Shared Environments
    Polo, Jorda
    Becerra, Yolanda
    Carrera, David
    Torres, Jordi
    Ayguade, Eduard
    Steinder, Malgorzata
    2014 14TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID), 2014, : 61 - 70
  • [33] A COMPARATIVE REVIEW OF JOB SCHEDULING FOR MAPREDUCE
    Yoo, Dongjin
    Sim, Kwang Mong
    2011 IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS, 2011, : 353 - 358
  • [34] TaskTracker Aware Scheduling for Hadoop MapReduce
    Manjaly, Jisha S.
    Chooralil, Varghese S.
    2013 THIRD INTERNATIONAL CONFERENCE ON ADVANCES IN COMPUTING AND COMMUNICATIONS (ICACC 2013), 2013, : 278 - 281
  • [35] Scheduling for response time in Hadoop MapReduce
    Dai, Xiangming
    Bensaou, Brahim
    2016 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS (ICC), 2016,
  • [36] Task scheduling for MapReduce in heterogeneous networks
    Jia Wang
    Xiaoping Li
    Cluster Computing, 2016, 19 : 197 - 210
  • [37] A REVIEW ON JOB SCHEDULING FOR HADOOP MAPREDUCE
    Kalia, Khushboo
    Gupta, Neeraj
    2017 INTERNATIONAL CONFERENCE ON NEXT GENERATION COMPUTING AND INFORMATION SYSTEMS (ICNGCIS), 2017, : 75 - 79
  • [38] Classification Framework of MapReduce Scheduling Algorithms
    Tiwari, Nidhi
    Sarkar, Santonu
    Bellur, Umesh
    Indrawan, Maria
    ACM COMPUTING SURVEYS, 2015, 47 (03)
  • [39] Scheduling Divisible Loads with Nonlinear Communication Time
    Wang, Kai
    Robertazzi, Thomas G.
    IEEE TRANSACTIONS ON AEROSPACE AND ELECTRONIC SYSTEMS, 2015, 51 (03) : 2479 - U955
  • [40] Divisible Load Scheduling in Systems with Limited Memory
    M. Drozdowski
    P. Wolniewicz
    Cluster Computing, 2003, 6 (1) : 19 - 29
12345