AN ANALYTICAL PERFORMANCE MODEL OF MAPREDUCE

被引:0
|
作者
Yang, Xiao [1 ]
Sun, Jianling [1 ]
机构
[1] Zhejiang Univ, Dept Comp Sci & Technol, Hangzhou, Zhejiang, Peoples R China
关键词
performance model; MapReduce; distributed computing;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
MapReduce is a distributed computing framework. Its application in distributed systems is a rapidly emerging field. Although this framework can leverage clusters to improve computing performance, tuning it is still challenging. Most current works related to MapReduce performance are based on system monitoring and simulation, and lack analytical performance models. In this paper, we propose a simple and general MapReduce performance model for better understanding the impact of each component on overall. program performance, and verify it in a small cluster. The results indicate that our model can predict the performance of MapReduce system and its relation to the configuration. According to our model, performance can be. improved. significantly by modifying Map split granularity and number of reducers without modifying the framework. The model also points out potential bottlenecks of the framework and future improvement for better performance.
引用
收藏
页码:306 / 310
页数:5
相关论文
共 50 条
  • [1] Analytical Performance Models for MapReduce Workloads
    Vianna, Emanuel
    Comarela, Giovanni
    Pontes, Tatiana
    Almeida, Jussara
    Almeida, Virgilio
    Wilkinson, Kevin
    Kuno, Harumi
    Dayal, Umeshwar
    [J]. INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING, 2013, 41 (04) : 495 - 525
  • [2] Analytical Performance Models for MapReduce Workloads
    Emanuel Vianna
    Giovanni Comarela
    Tatiana Pontes
    Jussara Almeida
    Virgílio Almeida
    Kevin Wilkinson
    Harumi Kuno
    Umeshwar Dayal
    [J]. International Journal of Parallel Programming, 2013, 41 : 495 - 525
  • [3] Performance comparison under failures of MPI and MapReduce: An analytical approach
    Jin, Hui
    Sun, Xian-He
    [J]. FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2013, 29 (07): : 1808 - 1815
  • [4] Performance Prediction Model in Heterogeneous MapReduce Environment
    Fan, Yuanquan
    Wu, Weiguo
    Xu, Yunlong
    Cao, Yangjie
    Li, Qian
    Cui, Jinhua
    Duan, Zhangfeng
    [J]. 2014 IEEE INTERNATIONAL CONFERENCE ON COMPUTER AND INFORMATION TECHNOLOGY (CIT), 2014, : 240 - 245
  • [5] Mapreduce performance model for Hadoop 2.x
    Glushkova, Dada
    Jovanovic, Petar
    Abello, Alberto
    [J]. INFORMATION SYSTEMS, 2019, 79 : 32 - 43
  • [6] Parameterizable benchmarking framework for designing a MapReduce performance model
    Zhang, Zhuoyao
    Cherkasova, Ludmila
    Loo, Boon Thau
    [J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2014, 26 (12): : 2005 - 2026
  • [7] Application and Performance Optimization of MapReduce Model in Image Segmentation
    Li, Maozhen
    Meng, Lu
    Wang, Jiaying
    Jin, Yong
    Hu, Binyu
    Chen, Youxing
    [J]. IEEE ACCESS, 2020, 8 : 31835 - 31844
  • [8] Performance Model of MapReduce Iterative Applications for Hybrid Cloud Bursting
    Clemente-Castello, Francisco J.
    Nicolae, Bogdan
    Mayo, Rafael
    Carlos Fernandez, Juan
    [J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2018, 29 (08) : 1794 - 1807
  • [9] On the Performance Projectability of MapReduce
    Xie, Di
    Hu, Y. Charlie
    Kompella, Ramana Rao
    [J]. 2012 IEEE 4TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING TECHNOLOGY AND SCIENCE (CLOUDCOM), 2012,
  • [10] An analytical model of TCP performance
    Kassa, Debessay Fesehaye
    Wittevrongel, Sabine
    [J]. 2006 IEEE INTERNATIONAL PERFORMANCE COMPUTING AND COMMUNICATIONS CONFERENCE, VOLS 1 AND 2, 2006, : 357 - +