Characterization and Modeling of PIDX Parallel I/O for Performance Optimization

被引：10

作者：

Kumar, Sidharth ^{[1
]}

Saha, Avishek ^{[1
]}

Vishwanath, Venkatram ^{[2
]}

Carns, Philip ^{[2
]}

Schmidt, John A. ^{[1
]}

Scorzelli, Giorgio ^{[1
]}

Kolla, Hemanth ^{[4
]}

Grout, Ray ^{[6
]}

Latham, Robert ^{[2
]}

Ross, Robert ^{[2
]}

Papka, Michael E. ^{[2
,4
,5
]}

Chen, Jacqueline ^{[4
]}

Pascucci, Valerio ^{[1
,3
]}

机构：

[1] Univ Utah, Sci Comp & Imaging Inst, Salt Lake City, UT 84112 USA

[2] Argonne Natl Lab, Argonne, IL USA

[3] Pacific Northwest Natl Lab, Richland, WA USA

[4] Sandia Natl Labs, Livermore, CA USA

[5] No Illinois Univ, De Kalb, IL 60115 USA

[6] Natl Renewable Energy Lab, Golden, CO USA

来源：

2013 INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS (SC) | 2013年

关键词：

I/O & Network Characterization; Performance Modeling;

D O I：

10.1145/2503210.2503252

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

Parallel I/O library performance can vary greatly in response to user-tunable parameter values such as aggregator count, file count, and aggregation strategy. Unfortunately, manual selection of these values is time consuming and dependent on characteristics of the target machine, the underlying file system, and the dataset itself. Some characteristics, such as the amount of memory per core, can also impose hard constraints on the range of viable parameter values. In this work we address these problems by using machine learning techniques to model the performance of the PIDX parallel I/O library and select appropriate tunable parameter values. We characterize both the network and I/O phases of PIDX on a Cray XE6 as well as an IBM Blue Gene/P system. We use the results of this study to develop a machine learning model for parameter space exploration and performance prediction.

引用

页数：12

共 50 条

[31] Doppio: I/O-Aware Performance Analysis, Modeling and Optimization for In-Memory Computing Framework
Zhou, Peipei
Ruan, Zhenyuan
Fang, Zhenman
Shand, Megan
Roazen, David
Cong, Jason
2018 IEEE INTERNATIONAL SYMPOSIUM ON PERFORMANCE ANALYSIS OF SYSTEMS AND SOFTWARE (ISPASS), 2018, : 22 - 32
[32] Memcached Optimization on High Performance I/O Technology
An Z.
Du H.
Li Q.
Huo Z.
Ma J.
Jisuanji Yanjiu yu Fazhan/Computer Research and Development, 2018, 55 (04): : 864 - 874
[33] PARALLEL I/O AS A PARALLEL APPLICATION
MOYER, SA
SUNDERAM, VS
INTERNATIONAL JOURNAL OF SUPERCOMPUTER APPLICATIONS AND HIGH PERFORMANCE COMPUTING, 1995, 9 (02): : 95 - 107
[34] Automated tuning of parallel I/O systems: An approach to portable I/O performance for scientific applications
Chen, Y
Winslett, M
IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 2000, 26 (04) : 362 - 383
[35] I/O Performance Modeling of Virtualized Storage Systems
Noorshams, Qais
Rostami, Kiana
Kounev, Samuel
Tuma, Petr
Reussner, Ralf
2013 IEEE 21ST INTERNATIONAL SYMPOSIUM ON MODELING, ANALYSIS & SIMULATION OF COMPUTER AND TELECOMMUNICATION SYSTEMS (MASCOTS 2013), 2013, : 121 - +
[36] Characterization of I/O requirements in a massively parallel shelf sea model
Lockey, P
Proctor, R
James, ID
INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS, 1998, 12 (03): : 320 - 332
[37] Characterization of I/O requirements in a massively parallel shelf sea model
Proudman Oceanographic Lab, Merseyside, United Kingdom
Int J High Perform Comput Appl, 3 (320-332):
[38] PERFORMANCE ESTIMATING FOR PARALLEL PERFORMANCE OPTIMIZATION
DUNLOP, AN
HEY, AJG
NICOLE, DA
PRITCHARD, DJ
SUPERCOMPUTER, 1995, 11 (04): : 19 - 30
[39] Performance characterization of irregular I/O at the extreme scale
Herbein, S.
McDaniel, S.
Podhorszki, N.
Logan, J.
Klasky, S.
Taufer, M.
PARALLEL COMPUTING, 2016, 51 : 17 - 36
[40] Using object based files for high performance parallel I/O
Logan, Jeremy
Dickens, Phillip M.
IDAACS 2007: PROCEEDINGS OF THE 4TH IEEE WORKSHOP ON INTELLIGENT DATA ACQUISITION AND ADVANCED COMPUTING SYSTEMS: TECHNOLOGY AND APPLICATIONS, 2007, : 149 - +

← 1 2 3 4 5 →