Quantification and analysis of performance fluctuation in distributed file system

被引:0
|
作者
Zhang, Yuchen [1 ]
Zhang, Xiao [1 ]
Yu, Jinyang [1 ]
Shi, Yulong [1 ]
Su, Linxuan [1 ]
机构
[1] Northwestern Polytech Univ, IT Acad, Dong Xiang Rd, Xian 710129, Shaanxi, Peoples R China
关键词
Distributed file system; Performance fluctuations; Quantitative analysis; Ceph;
D O I
10.1007/s10586-023-04141-4
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Performance is a key requirement in quality of service for cloud storage system users. Cloud storage system is complex, so the performance results may be different even under exact same experimental conditions. Many benchmarks are developed to measure performance, but there are only several research focus on performance fluctuations. In this paper, we present a generic method and two metrics to measure the performance fluctuations of a distributed file system. It analyzes the IO path and finds related factors such as storage media, network, configurations and software components. We tested the performance fluctuations of Ceph under different storage media HDD, SSD, and different workloads Webserver, Fileserver, and Mailserver. We adjust the software configuration and software components to reduce the performance fluctuations. By using Filestore, it reduces 85.4% performance fluctuations than Filestore. By using EXT4 instead of XFS, it reduces performance fluctuations by 33.2%. But reducing performance fluctuations of lower components does not always reduce the performance fluctuation of distributed file system. We reduced the performance fluctuation of the local file system by 80.7%, but the performance fluctuation of DFS by 38.0%.
引用
收藏
页码:3149 / 3162
页数:14
相关论文
共 50 条
  • [31] DISTRIBUTED MULTIMEDIA FILE SYSTEM
    Radovici, Alexandru
    Miu, Tudor
    Dragoi, George
    [J]. ANNALS OF DAAAM FOR 2009 & PROCEEDINGS OF THE 20TH INTERNATIONAL DAAAM SYMPOSIUM, 2009, 20 : 593 - 594
  • [32] Quantitative Analysis of File System Performance on NVM
    Choi, Gunhee
    Kim, Seungboo
    Choi, Jongmoo
    [J]. 2018 7TH IEEE NON-VOLATILE MEMORY SYSTEMS AND APPLICATIONS SYMPOSIUM (NVMSA 2018), 2018, : 109 - 110
  • [33] Performance Evaluation and Tuning for MapReduce Computing in Hadoop Distributed File System
    Kim, Jongyeop
    Kumar, Ashwin T. K.
    George, K. M.
    Park, Nohpill
    [J]. PROCEEDINGS 2015 IEEE INTERNATIONAL CONFERENCE ON INDUSTRIAL INFORMATICS (INDIN), 2015, : 62 - 68
  • [34] Erasure Code of Small File in a Distributed File System
    Chen, Xinhai
    Liu, Jie
    Xie, Peizhen
    [J]. PROCEEDINGS OF 2017 3RD IEEE INTERNATIONAL CONFERENCE ON COMPUTER AND COMMUNICATIONS (ICCC), 2017, : 2549 - 2554
  • [35] Optimisation to the execution performance of grid job based on distributed file system
    Hu, Liang D.
    Lin, Lin
    Zhao, Jia
    Che, Xilong
    Wei, Xiaohui
    [J]. INTERNATIONAL JOURNAL OF PARALLEL EMERGENT AND DISTRIBUTED SYSTEMS, 2012, 27 (02) : 109 - 121
  • [36] Computer Performance Determination System Based on Big Data Distributed File
    Lu, Kong
    [J]. CYBER SECURITY INTELLIGENCE AND ANALYTICS, 2020, 928 : 877 - 884
  • [37] Investigation of Replication Factor for Performance Enhancement in the Hadoop Distributed File System
    Ciritoglu, Hilmi Egemen
    de Almeida, Leandro Batista
    de Almeida, Eduardo Cunha
    Buda, Teodora Sandra
    Murphy, John
    Thorpe, Christina
    [J]. COMPANION OF THE 2018 ACM/SPEC INTERNATIONAL CONFERENCE ON PERFORMANCE ENGINEERING (ICPE '18), 2018, : 135 - 140
  • [38] A PERFORMANCE COMPARISON OF DEADLOCK PREVENTION AND DETECTION ALGORITHMS IN A DISTRIBUTED FILE SYSTEM
    HAC, A
    JIN, XW
    SOO, JH
    [J]. EIGHTH ANNUAL INTERNATIONAL PHOENIX CONFERENCE ON COMPUTERS AND COMMUNICATIONS: 1989 CONFERENCE PROCEEDINGS, 1989, : 473 - 477
  • [39] Research of the Strategy to Improve the Hadoop Distributed File System Downloading Performance
    Ge Junwei
    Jiang Renxiang
    [J]. ELECTRONIC INFORMATION AND ELECTRICAL ENGINEERING, 2012, 19 : 244 - 247
  • [40] Evaluating ParFiSys: A high-performance parallel and distributed file system
    Perez, F
    Carretero, J
    Garcia, F
    DeMiguel, P
    Alonso, L
    [J]. JOURNAL OF SYSTEMS ARCHITECTURE, 1997, 43 (08) : 533 - 542