I/O characterization and performance evaluation of large-scale storage architectures for heterogeneous workloads

被引：0

作者：

Kogiou, Olga ^{[1
]}

Devarajan, Hariharan ^{[2
]}

Wang, Chen ^{[2
]}

Yu, Weikuan ^{[1
]}

Mohror, Kathryn ^{[2
]}

机构：

[1] Florida State Univ, Tallahassee, FL 32306 USA

[2] Lawrence Livermore Natl Lab, Livermore, CA 94550 USA

来源：

2023 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING WORKSHOPS, CLUSTER WORKSHOPS | 2023年

关键词：

VAST; IOR benchmark; HPC applications;

D O I：

10.1109/CLUSTERWorkshops61457.2023.00017

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

HPC systems traditionally supported compute-centric workloads. However, the increasing reliance on data has led to a shift towards data-dependent workloads. This transition has necessitated storage technologies that enable fast data sharing among workflow parts, but diverse I/O requirements demand tailored solutions. New HPC architectures incorporate specialized software layers like Datawarp, IME, and VAST. However, user-driven storage system selection may lead to improper choices. Our investigation compares VAST with GPFS and Lustre filesystems across multiple machines, measuring performance, scalability, and identifying suitable I/O behaviors. This work provides a guide for selecting the appropriate storage system to optimize data access based on user requirements.

引用

页码：44 / 45

页数：2

共 50 条

[41] A Simulator for Large-Scale Parallel Computer Architectures
Janssen, Curtis L.
Adalsteinsson, Helgi
Cranford, Scott
Kenny, Joseph P.
Pinar, Ali
Evensky, David A.
Mayo, Jackson
INTERNATIONAL JOURNAL OF DISTRIBUTED SYSTEMS AND TECHNOLOGIES, 2010, 1 (02) : 57 - 73
[42] Directions in modelling large-scale software architectures
Grundy, J
Hosking, J
INTERNATIONAL CONFERENCE ON SOFTWARE METHODS AND TOOLS, PROCEEDING, 2000, : 31 - 40
[43] A type-centric framework for specifying heterogeneous, large-scale, component-oriented, architectures
Jung, Georg
Hatcliff, John
SCIENCE OF COMPUTER PROGRAMMING, 2010, 75 (07) : 615 - 637
[44] Performance Analysis of the FCBSH Algorithm for Large-Scale Heterogeneous Computing Environments
Du, Xiaoli
Jiang, Changjun
Yin, Fei
JOURNAL OF ALGORITHMS & COMPUTATIONAL TECHNOLOGY, 2009, 3 (04) : 461 - 476
[45] Diagnosing Performance Issues for Large-Scale Microservice Systems With Heterogeneous Graph
Tao, Lei
Lu, Xianglin
Zhang, Shenglin
Luan, Jiaqi
Li, Yingke
Li, Mingjie
Li, Zeyan
Yu, Qingyang
Xie, Hucheng
Xu, Ruijie
Hu, Chenyuan
Yang, Canqun
Pei, Dan
IEEE TRANSACTIONS ON SERVICES COMPUTING, 2024, 17 (05) : 2223 - 2235
[46] Performance optimization of heterogeneous computing for large-scale dynamic graph data
Wang, Haifeng
Guo, Wenkang
Zhang, Ming
JOURNAL OF SUPERCOMPUTING, 2025, 81 (01):
[47] Large-Scale Heterogeneous Feature Embedding
Huang, Xiao
Song, Qingquan
Yang, Fan
Hu, Xia
THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FIRST INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / NINTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2019, : 3878 - 3885
[48] TraceTracker: Hardware/Software Co-Evaluation for Large-Scale I/O Workload Reconstruction
Kwon, Miryeong
Zhang, Jie
Park, Gyuyoung
Choi, Wonil
Donofrio, David
Shalf, John
Kandemir, Mahmut
Jung, Myoungsoo
PROCEEDINGS OF THE 2017 IEEE INTERNATIONAL SYMPOSIUM ON WORKLOAD CHARACTERIZATION (IISWC), 2017, : 87 - 96
[49] Large Sensor Network Architectures for Monitoring Large-Scale Structures
Zhang, D. C.
Narayanan, V.
Zheng, X. B.
Chung, H.
Banerjee, S.
Beard, S.
Li, I.
STRUCTURAL HEALTH MONITORING 2011: CONDITION-BASED MAINTENANCE AND INTELLIGENT STRUCTURES, VOL 1, 2011, : 421 - 431
[50] IOscope: A Flexible I/O Tracer for Workloads' I/O Pattern Characterization
Saif, Abdulqawi
Nussbaum, Lucas
Song, Ye-Qiong
HIGH PERFORMANCE COMPUTING, ISC HIGH PERFORMANCE 2018, 2018, 11203 : 103 - 116

← 1 2 3 4 5 →