Application Performance Analysis of Distributed File Systems under Cloud Computing Environment

被引：1

作者：

Zhao, Tiezhu ^{[1
]}

Zhang, Zusheng ^{[1
]}

Ao, Xin ^{[1
]}

机构：

[1] Dongguan Univ Technol, Coll Comp, Dongguan 523808, Peoples R China

来源：

2015 2ND INTERNATIONAL CONFERENCE ON INFORMATION SCIENCE AND CONTROL ENGINEERING ICISCE 2015 | 2015年

关键词：

Distributed file system; Hadoop; Lustre; Data-intensive application;

D O I：

10.1109/ICISCE.2015.41

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

The processing efficiency of data-intensive application on Hadoop with the general-purpose distributed file system such as Lustre, as the backend file system, is not clear. This paper focuses on the similarities and differences between Lustre and HDFS (Hadoop Distributed File System). We propose a Hadoop-Lustre platform and evaluate the performance differences of Lustre and HDFS by using a set of data-intensive computing benchmarks. Experimental results indicate Lustre can reach parity with HDFS, or even better than HDFS if the much faster network interconnect is available. It is necessary to study non-HDFS distributed file system to make up the performance lack of HDFS in some Map-Reduce-based application scenarios.

引用

页码：152 / 155

页数：4

共 50 条

[1] Integrating High Performance File Systems in a Cloud Computing Environment
Pan, Abhisek
Walters, John Paul
Pai, Vijay S.
Kang, Dong-In D.
Crago, Stephen P.
[J]. 2012 SC COMPANION: HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS (SCC), 2012, : 753 - 759
[2] The Enterprise Distributed File System Metadata Distribution in Cloud Computing Environment
Suo, Hui
Zhou, Gui-Xian
Liu, Zhuo-hua
[J]. 2016 INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND INFORMATION SECURITY (CSIS 2016), 2016, : 435 - 440
[3] Performing Initiative Data Prefetching in Distributed File Systems for Cloud Computing
Liao, Jianwei
Trahay, Francois
Xiao, Guoqiang
Li, Li
Ishikawa, Yutaka
[J]. IEEE TRANSACTIONS ON CLOUD COMPUTING, 2017, 5 (03) : 550 - 562
[4] Distributed Malware Detection based on Binary File Features in Cloud Computing Environment
Han, Xiaoguang
Sun, Jigang
Qu, Wu
Yao, Xuanxia
[J]. 26TH CHINESE CONTROL AND DECISION CONFERENCE (2014 CCDC), 2014, : 4083 - 4088
[5] A Data Management in a Private Cloud Storage Environment Utilizing High Performance Distributed File Systems
Soares, Tiago S.
Dantas, M. A. R.
de Macedo, Douglas D. J.
Bauer, Michael A.
[J]. 2013 IEEE 22ND INTERNATIONAL WORKSHOP ON ENABLING TECHNOLOGIES: INFRASTRUCTURE FOR COLLABORATIVE ENTERPRISES (WETICE), 2013, : 158 - 163
[6] Survey of Analysis of Performance in Cloud Computing Environment
Mishra, Abhishek
Ali, Nashet
Gupta, Avinash
[J]. 2017 IEEE INTERNATIONAL CONFERENCE ON POWER, CONTROL, SIGNALS AND INSTRUMENTATION ENGINEERING (ICPCSI), 2017, : 572 - 577
[7] Performance Monitoring and Analysis of Cloud Computing Environment
Khanghahi, Niloofar
Ravanmehr, Reza
[J]. 2013 FIRST INTERNATIONAL SCIENTIFIC CONFERENCE ON ELECTRICAL, COMMUNICATION, COMPUTER, POWER, AND CONTROL ENGINEERING (ICECCPCE'13), 2013, : 185 - 190
[8] Performance modeling and analysis of a distributed computing environment
Drakopoulos, E
[J]. THIRD IEEE SYMPOSIUM ON COMPUTERS AND COMMUNICATIONS, PROCEEDINGS, 1998, : 210 - 215
[9] Distributed Authentication in the Cloud Computing Environment
Liu, Yanzhu
Li, Zhi
Sun, Yuxia
[J]. ALGORITHMS AND ARCHITECTURES FOR PARALLEL PROCESSING, ICA3PP 2015, 2015, 9532 : 810 - 818
[10] Modeling of Distributed File Systems for Practical Performance Analysis
Wu, Yongwei
Ye, Feng
Chen, Kang
Zheng, Weimin
[J]. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2014, 25 (01) : 156 - 166

← 1 2 3 4 5 →