I/O characterization and performance evaluation of large-scale storage architectures for heterogeneous workloads

被引：0
|
作者：
Kogiou, Olga [1 ]
Devarajan, Hariharan [2 ]
Wang, Chen [2 ]
Yu, Weikuan [1 ]
Mohror, Kathryn [2 ]
机构：
[1] Florida State Univ, Tallahassee, FL 32306 USA
[2] Lawrence Livermore Natl Lab, Livermore, CA 94550 USA
来源：
2023 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING WORKSHOPS, CLUSTER WORKSHOPS | 2023年
关键词：
VAST; IOR benchmark; HPC applications;
D O I：
10.1109/CLUSTERWorkshops61457.2023.00017
中图分类号：
TP3 [计算技术、计算机技术];
学科分类号：
0812 ;
摘要：
HPC systems traditionally supported compute-centric workloads. However, the increasing reliance on data has led to a shift towards data-dependent workloads. This transition has necessitated storage technologies that enable fast data sharing among workflow parts, but diverse I/O requirements demand tailored solutions. New HPC architectures incorporate specialized software layers like Datawarp, IME, and VAST. However, user-driven storage system selection may lead to improper choices. Our investigation compares VAST with GPFS and Lustre filesystems across multiple machines, measuring performance, scalability, and identifying suitable I/O behaviors. This work provides a guide for selecting the appropriate storage system to optimize data access based on user requirements.
引用
收藏
页码：44 / 45
页数：2
相关论文
共 50 条
  • [1] Performance Characterization on Handling Large-Scale Partitionable Workloads on Heterogeneous Networked Compute Platforms
    Wang, Xiaoli
    Veeravalli, Bharadwaj
    IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2017, 28 (10) : 2925 - 2938
  • [2] A Study of Simulating Heterogeneous Workloads on Large-scale Interconnect Network
    Wang, Xin
    PROCEEDINGS OF THE 2023 ACM SIGSIM INTERNATIONAL CONFERENCE ON PRINCIPLES OF ADVANCED DISCRETE SIMULATION, ACMSIGSIM-PADS 2023, 2023, : 58 - 59
  • [3] A Reconfigurable Simulator for Large-scale Heterogeneous Multicore Architectures
    Meng, Jiayuan
    Skadron, Kevin
    IEEE INTERNATIONAL SYMPOSIUM ON PERFORMANCE ANALYSIS OF SYSTEMS AND SOFTWARE (ISPASS 2011), 2011, : 119 - 120
  • [4] Characterization of I/O Behaviors in Cloud Storage Workloads
    Zou, Qiang
    Zhu, Yifeng
    Chen, Jianxi
    Deng, Yuhui
    Qin, Xiao
    IEEE TRANSACTIONS ON COMPUTERS, 2023, 72 (10) : 2726 - 2739
  • [5] Accelerating Large-scale Image Retrieval on Heterogeneous Architectures with Spark
    Wang, Hanli
    Xiao, Bo
    Wang, Lei
    Wu, Jun
    MM'15: PROCEEDINGS OF THE 2015 ACM MULTIMEDIA CONFERENCE, 2015, : 1023 - 1026
  • [6] Strategic directions in storage I/O issues in large-scale computing
    Gibson, GA
    Vitter, JS
    Wilkes, J
    ACM COMPUTING SURVEYS, 1996, 28 (04) : 779 - 793
  • [7] Pelican: Power Scheduling for QoS in Large-scale Data Centers with Heterogeneous Workloads
    Luo, Bing
    Chen, Wei
    Liu, Xingxing
    Li, Xiaozhong
    Zhang, Lifei
    Shi, Weisong
    2019 TENTH INTERNATIONAL GREEN AND SUSTAINABLE COMPUTING CONFERENCE (IGSC), 2019,
  • [8] An In-Depth Analysis of Cloud Block Storage Workloads in Large-Scale Production
    Li, Jinhong
    Wang, Qiuping
    Lee, Patrick P. C.
    Shi, Chao
    2020 IEEE INTERNATIONAL SYMPOSIUM ON WORKLOAD CHARACTERIZATION (IISWC 2020), 2020, : 37 - 47
  • [9] Performance Prediction for Large-scale Heterogeneous Platforms
    Yasudo, Ryota
    Varbanescu, Ana L.
    Coutinho, Jose G. F.
    Luk, Wayne
    Amano, Hideharu
    PROCEEDINGS 26TH IEEE ANNUAL INTERNATIONAL SYMPOSIUM ON FIELD-PROGRAMMABLE CUSTOM COMPUTING MACHINES (FCCM 2018), 2018, : 220 - 220
  • [10] Performance Evaluation of NoC Architectures for Parallel Workloads
    Freitas, Henrique C.
    Alves, Marco A. Z.
    Schnorr, Lucas M.
    Navaux, Philippe O. A.
    2009 3RD ACM/IEEE INTERNATIONAL SYMPOSIUM ON NETWORKS-ON-CHIP, 2009, : 87 - 87
12345