I/O characterization and performance evaluation of large-scale storage architectures for heterogeneous workloads

被引：0
|
作者：
Kogiou, Olga [1 ]
Devarajan, Hariharan [2 ]
Wang, Chen [2 ]
Yu, Weikuan [1 ]
Mohror, Kathryn [2 ]
机构：
[1] Florida State Univ, Tallahassee, FL 32306 USA
[2] Lawrence Livermore Natl Lab, Livermore, CA 94550 USA
来源：
2023 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING WORKSHOPS, CLUSTER WORKSHOPS | 2023年
关键词：
VAST; IOR benchmark; HPC applications;
D O I：
10.1109/CLUSTERWorkshops61457.2023.00017
中图分类号：
TP3 [计算技术、计算机技术];
学科分类号：
0812 ;
摘要：
HPC systems traditionally supported compute-centric workloads. However, the increasing reliance on data has led to a shift towards data-dependent workloads. This transition has necessitated storage technologies that enable fast data sharing among workflow parts, but diverse I/O requirements demand tailored solutions. New HPC architectures incorporate specialized software layers like Datawarp, IME, and VAST. However, user-driven storage system selection may lead to improper choices. Our investigation compares VAST with GPFS and Lustre filesystems across multiple machines, measuring performance, scalability, and identifying suitable I/O behaviors. This work provides a guide for selecting the appropriate storage system to optimize data access based on user requirements.
引用
收藏
页码：44 / 45
页数：2
相关论文
共 50 条
  • [21] Enabling Serverless Deployment of Large-Scale AI Workloads
    Christidis, Angelos
    Moschoyiannis, Sotiris
    Hsu, Ching-Hsien
    Davies, Roy
    IEEE ACCESS, 2020, 8 : 70150 - 70161
  • [22] Decoupling Datacenter Studies from Access to Large-Scale Applications: A Modeling Approach for Storage Workloads
    Delimitrou, Christina
    Sankar, Sriram
    Vaid, Kushagra
    Kozyrakis, Christos
    2011 IEEE INTERNATIONAL SYMPOSIUM ON WORKLOAD CHARACTERIZATION (IISWC), 2011, : 51 - 60
  • [23] Selecting Subexpressions to Materialize for Dynamic Large-Scale Workloads
    Mouna, Mustapha Chaba
    Bellatreche, Ladjel
    Boustia, Narhimene
    BIG DATA ANALYTICS AND KNOWLEDGE DISCOVERY (DAWAK 2021), 2021, 12925 : 39 - 51
  • [24] Research on Performance Evaluation of Large-Scale Enterprise
    Gu Dan-dan
    2015 INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION, BIG DATA AND SMART CITY (ICITBS), 2016, : 661 - 664
  • [25] I/O performance analysis of machine learning workloads on leadership scale supercomputer
    Karimi, Ahmad Maroof
    Paul, Arnab K.
    Wang, Feiyi
    PERFORMANCE EVALUATION, 2022, 157
  • [26] Large-scale Performance Evaluation of e-Homecare Architectures Using the WS-NS Simulator
    Van Hoecke, S.
    Volckaert, B.
    Dhoedt, B.
    De Turck, F.
    METHODS OF INFORMATION IN MEDICINE, 2011, 50 (05) : 408 - 419
  • [27] AdapCK: Optimizing I/O for Checkpointing on Large-Scale High Performance Computing Systems
    Jia, Jie
    Liu, Yi
    Liu, Yanke
    Chen, Yifan
    Lin, Fang
    EURO-PAR 2024: PARALLEL PROCESSING, PT III, EURO-PAR 2024, 2024, 14803 : 342 - 355
  • [28] Towards HPC I/O Performance Prediction Through Large-scale Log Analysis
    Kim, Sunggon
    Sim, Alex
    Wu, Kesheng
    Byna, Suren
    Son, Yongseok
    Eom, Hyeonsang
    PROCEEDINGS OF THE 29TH INTERNATIONAL SYMPOSIUM ON HIGH-PERFORMANCE PARALLEL AND DISTRIBUTED COMPUTING, HPDC 2020, 2020, : 77 - 88
  • [29] How Different are the Cloud Workloads? Characterizing Large-Scale Private and Public Cloud Workloads
    Qin, Xiaoting
    Ma, Minghua
    Zhao, Yuheng
    Zhang, Jue
    Du, Chao
    Liu, Yudong
    Parayil, Anjaly
    Bansal, Chetan
    Rajmohan, Saravan
    Goiri, Inigo
    Cortez, Eli
    Qin, Si
    Lin, Qingwei
    Zhang, Dongmei
    2023 53RD ANNUAL IEEE/IFIP INTERNATIONAL CONFERENCE ON DEPENDABLE SYSTEMS AND NETWORKS, DSN, 2023, : 522 - 530
  • [30] Large-scale storage of hydrogen
    Andersson, Joakim
    Gronkvist, Stefan
    INTERNATIONAL JOURNAL OF HYDROGEN ENERGY, 2019, 44 (23) : 11901 - 11919
12345