Straggler Mitigation in Hadoop MapReduce Framework: A Review

被引:0
|
作者
Ajibade, Lukuman Saheed [1 ]
Abu Bakar, Kamalrulnizam [1 ]
Aliyu, Ahmed [2 ]
Danish, Tasneem [3 ]
机构
[1] Univ Teknol Malaysia, Sch Comp, Johor Baharu, Malaysia
[2] Bauchi State Univ, Dept Math, Gadau, Nigeria
[3] Carleton Univ Canada, Syst & Comp Engn Dept, Ottawa, ON, Canada
关键词
Big data; blacklisting execution; Hadoop; MapReduce; spark; speculative execution; straggler; SKEWED DATA; ALGORITHMS;
D O I
10.14569/IJACSA.2022.01308101
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Processing huge and complex data to obtain useful information is challenging, even though several big data processing frameworks have been proposed and further enhanced. One of the prominent big data processing frameworks is MapReduce. The main concept of MapReduce framework relies on distributed and parallel processing. However, MapReduce framework is facing serious performance degradations due to the slow execution of certain tasks type called stragglers. Failing to handle stragglers causes delay and affects the overall job execution time. Meanwhile, several straggler reduction techniques have been proposed to improve the MapReduce performance. This study provides a comprehensive and qualitative review of the different existing straggler mitigation solutions. In addition, a taxonomy of the available straggler mitigation solutions is presented. Critical research issues and future research directions are identified and discussed to guide researchers and scholars.
引用
收藏
页码:870 / 878
页数:9
相关论文
共 50 条
  • [31] An overview of the Hadoop/MapReduce/HBase framework and its current applications in bioinformatics
    Ronald C Taylor
    BMC Bioinformatics, 11
  • [32] ST-Hadoop: a MapReduce framework for spatio-temporal data
    Louai Alarabi
    Mohamed F. Mokbel
    Mashaal Musleh
    GeoInformatica, 2018, 22 : 785 - 813
  • [33] Optimizing the Hadoop MapReduce Framework with high-performance storage devices
    Moon, Sangwhan
    Lee, Jaehwan
    Sun, Xiling
    Kee, Yang-suk
    JOURNAL OF SUPERCOMPUTING, 2015, 71 (09): : 3525 - 3548
  • [34] Optimizing the Hadoop MapReduce Framework with high-performance storage devices
    Sangwhan Moon
    Jaehwan Lee
    Xiling Sun
    Yang-suk Kee
    The Journal of Supercomputing, 2015, 71 : 3525 - 3548
  • [35] Weighted Finite Automata based Image Compression on Hadoop MapReduce Framework
    Raju, U. S. N.
    Sandeep, Irlanki
    Karthik, Nattam Sai
    Praveen, Rayapudi Siva
    Sachan, Mayank Singh
    2015 IEEE INTERNATIONAL CONGRESS ON BIG DATA - BIGDATA CONGRESS 2015, 2015, : 653 - 656
  • [36] Performance Comparison of Distributed Pattern Matching Algorithms on Hadoop MapReduce Framework
    Sona, C. P.
    Mulerikkal, Jaison Paul
    MOBILE NETWORKS AND MANAGEMENT (MONAMI 2017), 2018, 235 : 45 - 55
  • [37] DPro-SM - A distributed framework for proactive straggler mitigation using LSTM
    Ravikumar, Aswathy
    Sriraman, Harini
    HELIYON, 2024, 10 (01)
  • [38] A Comparative Study on Improving Straggler Tasks in Hadoop
    Hussien, Gehad K.
    Khafagy, Mohamed H.
    Ibrahim, Mohamed H.
    Kaseb, Mostafa R.
    INFORMATION SYSTEMS AND TECHNOLOGIES, WORLDCIST 2022, VOL 1, 2022, 468 : 52 - 65
  • [39] An Optimized Straggler Mitigation Framework for Large-Scale Distributed Computing Systems
    Said, Samar A.
    Habashy, Shahira M.
    Salem, Sameh A.
    Saad, Elsayed M.
    IEEE ACCESS, 2022, 10 : 97075 - 97088
  • [40] An Optimized Straggler Mitigation Framework for Large-Scale Distributed Computing Systems
    Said, Samar A.
    Habashy, Shahira M.
    Salem, Sameh A.
    Saad, Elsayed M.
    IEEE Access, 2022, 10 : 97075 - 97088