AN ANALYTICAL PERFORMANCE MODEL OF MAPREDUCE

被引:0
|
作者
Yang, Xiao [1 ]
Sun, Jianling [1 ]
机构
[1] Zhejiang Univ, Dept Comp Sci & Technol, Hangzhou, Zhejiang, Peoples R China
关键词
performance model; MapReduce; distributed computing;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
MapReduce is a distributed computing framework. Its application in distributed systems is a rapidly emerging field. Although this framework can leverage clusters to improve computing performance, tuning it is still challenging. Most current works related to MapReduce performance are based on system monitoring and simulation, and lack analytical performance models. In this paper, we propose a simple and general MapReduce performance model for better understanding the impact of each component on overall. program performance, and verify it in a small cluster. The results indicate that our model can predict the performance of MapReduce system and its relation to the configuration. According to our model, performance can be. improved. significantly by modifying Map split granularity and number of reducers without modifying the framework. The model also points out potential bottlenecks of the framework and future improvement for better performance.
引用
收藏
页码:306 / 310
页数:5
相关论文
共 50 条
  • [11] A Combined Analytical Modeling Machine Learning Approach for Performance Prediction of MapReduce Jobs in Cloud Environment
    Ataie, Ehsan
    Gianniti, Eugenio
    Ardagna, Danilo
    Movaghar, Ali
    [J]. PROCEEDINGS OF 2016 18TH INTERNATIONAL SYMPOSIUM ON SYMBOLIC AND NUMERIC ALGORITHMS FOR SCIENTIFIC COMPUTING (SYNASC), 2016, : 431 - 439
  • [12] Model Driven Performance Simulation of Cloud Provisioned Hadoop MapReduce Applications
    Alipour, Hanieh
    Liu, Yan
    Hamou-Lhadj, Abdelwahab
    Gorton, Ian
    [J]. 2016 IEEE/ACM 8TH INTERNATIONAL WORKSHOP ON MODELING IN SOFTWARE ENGINEERING (MISE), 2016, : 48 - 54
  • [13] Towards Optimization of RDF Analytical Queries on MapReduce
    Ravindra, Padmashree
    [J]. 2014 IEEE 30TH INTERNATIONAL CONFERENCE ON DATA ENGINEERING WORKSHOPS (ICDEW), 2014, : 335 - 339
  • [14] High performance parallel evolutionary algorithm model based on MapReduce framework
    Du, Xin
    Ni, Youcong
    Yao, Zhiqiang
    Xiao, Ruliang
    Xie, Datong
    [J]. INTERNATIONAL JOURNAL OF COMPUTER APPLICATIONS IN TECHNOLOGY, 2013, 46 (03) : 290 - 295
  • [15] A Model of Computation for MapReduce
    Karloff, Howard
    Suri, Siddharth
    Vassilvitskii, Sergei
    [J]. PROCEEDINGS OF THE TWENTY-FIRST ANNUAL ACM-SIAM SYMPOSIUM ON DISCRETE ALGORITHMS, 2010, 135 : 938 - 948
  • [16] Analytical performance model for disk drives
    Kaddeche, H
    Beylot, AL
    Becker, M
    [J]. INTERNATIONAL SOCIETY FOR COMPUTERS AND THEIR APPLICATIONS 10TH INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED COMPUTING SYSTEMS, 1997, : 16 - 22
  • [17] Analytical model of hyperspectral system performance
    Kerekes, JP
    Baum, JE
    Farrar, KE
    [J]. INFRARED IMAGING SYSTEMS: DESIGN, ANALYSIS, MODELING, AND TESTING X, 1999, 3701 : 155 - 166
  • [18] An analytical performance model for the spidergon NoC
    Moadeli, Mahmoud
    Shahrabi, Ali
    Vanderbauwhede, Wim
    Ould-Khaoua, Mohamed
    [J]. 21ST INTERNATIONAL CONFERENCE ON ADVANCED NETWORKING AND APPLICATIONS, PROCEEDINGS, 2007, : 1014 - +
  • [19] On the Performance of MapReduce: A Stochastic Approach
    Ahmed, Sarker Tanzir
    Loguinov, Dmitri
    [J]. 2014 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2014, : 49 - 54
  • [20] Performance Analysis Using Petri Net Based MapReduce Model in Heterogeneous Clusters
    Cheng, Sheng-Tzong
    Wang, Hsi-Chuan
    Chen, Yin-Jun
    Chen, Chen-Fei
    [J]. ADVANCES IN WEB-BASED LEARNING, 2015, 8390 : 170 - 179