Optimizing Performance of Hadoop with Parameter Tuning

被引：3

作者：

Chen, Xiang ^{[1
]}

Liang, Yi ^{[1
]}

Li, Guang-Rui ^{[2
]}

Chen, Cheng ^{[1
]}

Liu, Si-Yu ^{[1
]}

机构：

[1] Beijing Univ Technol, Fac Informat Technol, Beijing, Peoples R China

[2] China Mobile Software Technol Co Ltd, 10 Bldg,Suzhou Software Pk,78, Suzhou, Peoples R China

来源：

4TH ANNUAL INTERNATIONAL CONFERENCE ON INFORMATION TECHNOLOGY AND APPLICATIONS (ITA 2017) | 2017年 / 12卷

关键词：

D O I：

10.1051/itmconf/20171203040

中图分类号：

TP39 [计算机的应用];

学科分类号：

081203 ; 0835 ;

摘要：

Optimizing Hadoop with the parameter tuning is an effective way to greatly improve the performance, but it usually costs too much time to identify the optimal parameters configuration because there are many parameters. Users are always blindly adjust too many parameters and are sometimes confused about which one could be changed at a higher-priority. To make optimization easier, classifying the parameter based on different applications will be helpful. In this paper, we will introduce a method that can classify these parameters in order that users can optimize performance more quickly and effectively for different applications.

引用

页数：4

共 50 条

[31] OPTIMIZING HADOOP DATA LOCALITY: PERFORMANCE ENHANCEMENT STRATEGIES IN HETEROGENEOUS COMPUTING ENVIRONMENTS
School of Electrical and Computer Engineering, Yeosu Campus, Chonnam National University, 59626, Korea, Republic of
Scalable Comput. Pract. Exp., 6 (4558-4575):
[32] Repairing and Optimizing Hadoop hashCode Implementations
Kocsis, Zoltan A.
Neumann, Geoff
Swan, Jerry
Epitropakis, Michael G.
Brownlee, Alexander E. I.
Haraldsson, Sami O.
Bowles, Edward
SEARCH-BASED SOFTWARE ENGINEERING, 2014, 8636 : 259 - 264
[33] CoTuner: A Hierarchical Learning Framework for Coordinately Optimizing Resource Partitioning and Parameter Tuning
Yang, Tiannuo
Chen, Ruobing
Li, Yusen
Liu, Xiaoguang
Wang, Gang
PROCEEDINGS OF THE 52ND INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING, ICPP 2023, 2023, : 317 - 326
[34] Piranha: Optimizing Short Jobs in Hadoop
Elmeleegy, Khaled
PROCEEDINGS OF THE VLDB ENDOWMENT, 2013, 6 (11): : 985 - 996
[35] Performance Analysis and Parameter Optimizing Rules of LT Codes
Guo Chunmei
Bi Xueyao
CHINA COMMUNICATIONS, 2010, 7 (04) : 103 - 107
[36] Automatic parameter tuning framework for performance diagnosis report
Uchiumi, Tetsuya
Saitoh, Yuji
Watanabe, Yukihiro
2019 20TH ASIA-PACIFIC NETWORK OPERATIONS AND MANAGEMENT SYMPOSIUM (APNOMS), 2019,
[37] Identifying Performance Bottlenecks based on the Local Parameter Tuning
Zenmyo, Teruyoshi
ACM/IEEE SIXTH INTERNATIONAL CONFERENCE ON AUTONOMIC COMPUTING AND COMMUNICATIONS (ICAC '09), 2009, : 53 - 54
[38] Parameter tuning of big data platforms for performance optimization
Pattanshetti, Tanuja
Attar, Vahida
JOURNAL OF INFORMATION & OPTIMIZATION SCIENCES, 2020, 41 (02): : 403 - 410
[39] Enhancing Performance of PSO with Automatic Parameter Tuning Technique
Tewolde, Girma S.
Hanna, Darrin M.
Haskell, Richard E.
2009 IEEE SWARM INTELLIGENCE SYMPOSIUM, 2009, : 67 - +
[40] Induction motor parameter tuning for high performance drives
Seok, JK
Sul, SK
CONFERENCE RECORD OF THE 1998 IEEE INDUSTRY APPLICATIONS CONFERENCE, VOLS 1-3, 1998, : 633 - 639

← 1 2 3 4 5 →