Optimizing Performance of Hadoop with Parameter Tuning

被引：3
|
作者：
Chen, Xiang [1 ]
Liang, Yi [1 ]
Li, Guang-Rui [2 ]
Chen, Cheng [1 ]
Liu, Si-Yu [1 ]
机构：
[1] Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China
[2] China Mobile Software Technol Co Ltd, 10 Bldg,Suzhou Software Pk,78, Suzhou, Peoples R China
来源：
4TH ANNUAL INTERNATIONAL CONFERENCE ON INFORMATION TECHNOLOGY AND APPLICATIONS (ITA 2017) | 2017年 / 12卷
关键词：
D O I：
10.1051/itmconf/20171203040
中图分类号：
TP39 [计算机的应用];
学科分类号：
081203 ; 0835 ;
摘要：
Optimizing Hadoop with the parameter tuning is an effective way to greatly improve the performance, but it usually costs too much time to identify the optimal parameters configuration because there are many parameters. Users are always blindly adjust too many parameters and are sometimes confused about which one could be changed at a higher-priority. To make optimization easier, classifying the parameter based on different applications will be helpful. In this paper, we will introduce a method that can classify these parameters in order that users can optimize performance more quickly and effectively for different applications.
引用
收藏
页数：4
相关论文
共 50 条
  • [41] Impact of Parameter Tuning for Optimizing Deep Neural Network Models for Predicting Software Faults
    Gupta, Mansi
    Rajnish, Kumar
    Bhattacharjee, Vandana
    SCIENTIFIC PROGRAMMING, 2021, 2021
  • [42] Optimizing Sort in Hadoop Using Replacement Selection
    Dusso, Pedro Martins
    Sauer, Caetano
    Haerder, Theo
    ADVANCES IN DATABASES AND INFORMATION SYSTEMS, ADBIS 2015, 2015, 9282 : 365 - 379
  • [43] Optimizing data placement in heterogeneous Hadoop clusters
    Runqun Xiong
    Junzhou Luo
    Fang Dong
    Cluster Computing, 2015, 18 : 1465 - 1480
  • [44] Optimizing Hadoop Framework for Solid State Drives
    Hong, Jaeki
    Han, Chihye
    Li, Liang
    Jin, Bingxu
    Yang, Qichao
    Yang, Zilong
    2016 IEEE INTERNATIONAL CONGRESS ON BIG DATA - BIGDATA CONGRESS 2016, 2016, : 9 - 17
  • [45] Optimizing data placement in heterogeneous Hadoop clusters
    Xiong, Runqun
    Luo, Junzhou
    Dong, Fang
    CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2015, 18 (04): : 1465 - 1480
  • [46] Towards an Ontology-Based Semantic Approach to Tuning Parameters to Improve Hadoop Application Performance
    Bonifacio, Ailton
    Menolli, Andre
    Silva, Fabiano
    INFORMATION TECHNOLOGY IN INDUSTRY, 2014, 2 (02): : 56 - 61
  • [47] Improving performance of logical qubits by parameter tuning and topology compensation
    Raymond, Jack
    Ndiaye, Ndiame
    Rayaprolu, Gautam
    King, Andrew D.
    IEEE INTERNATIONAL CONFERENCE ON QUANTUM COMPUTING AND ENGINEERING (QCE20), 2020, : 295 - 305
  • [48] Induction motor parameter tuning for high-performance drives
    Seok, JK
    Sul, SK
    IEEE TRANSACTIONS ON INDUSTRY APPLICATIONS, 2001, 37 (01) : 35 - 41
  • [49] Improving Simulated Annealing Performance by means of Automatic Parameter Tuning
    Cabrera-Guerrero, Pablo
    Guerrero, Guillermo
    Vega, Jorge
    Johnson, Franklin
    STUDIES IN INFORMATICS AND CONTROL, 2015, 24 (04): : 419 - 426
  • [50] Parameter Tuning Boosts Performance of Variation Operators in Multiobjective Optimization
    Wessing, Simon
    Beume, Nicola
    Rudolph, Guenter
    Naujoks, Boris
    PARALLEL PROBLEMS SOLVING FROM NATURE - PPSN XI, PT I, 2010, 6238 : 728 - 737
12345