Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing

被引：1

作者：

Cappello, Franck ^{[1
]}

Acosta, Mario ^{[10
]}

Agullo, Emmanuel ^{[9
]}

Anzt, Hartwig ^{[12
]}

Calhoun, Jon ^{[7
]}

Di, Sheng ^{[2
]}

Giraud, Luc ^{[9
]}

Gruetzmacher, Thomas ^{[12
]}

Jin, Sian ^{[4
]}

Sano, Kentaro ^{[8
]}

Sato, Kento ^{[8
]}

Singh, Amarjit ^{[8
]}

Tao, Dingwen ^{[5
]}

Tian, Jiannan ^{[6
]}

Ueno, Tomohiro ^{[8
]}

Underwood, Robert ^{[3
]}

Vivien, Frederic ^{[9
]}

Yepes, Xavier ^{[11
]}

Kazutomo, Yoshii ^{[1
]}

Zhang, Boyuan ^{[6
]}

机构：

[1] Argonne Natl Lab, Lemont, IL 60439 USA

[2] Argonne Natl Lab, Math & Comp Sci MCS Div, Lemont, IL USA

[3] Argonne Natl Lab, Math & Comp Sci Div, Lemont, IL USA

[4] Indiana Univ, Bloomington, IN USA

[5] Indiana Univ, HighPerformance Data Analyt & Comp Lab, Bloomington, IN USA

[6] Indiana Univ, Intelligent Syst Engn, Bloomington, IN USA

[7] Clemson Univ, Holcombe Dept Elect & Comp Engn, Clemson, SC USA

[8] RIKEN, Ctr Computat Sci, Kobe, Japan

[9] Natl Res Inst Comp & Automat, Lyon, France

[10] Barcelona Supercomp Ctr, Earth Sci Dept, Computat Grp, Barcelona, Spain

[11] Barcelona Supercomp Ctr, Barcelona, Spain

[12] Karlsruhe Inst Technol, Karlsruhe, Germany

来源：

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE | 2025年 / 163卷

基金：

美国国家科学基金会;

关键词：

Lossy compression; Scientific data; Compression for AI; GPU acceleration; I/O scheduling; EARTH SYSTEM MODEL; GMRES; SIMULATION; I/O;

D O I：

10.1016/j.future.2024.05.022

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

The Joint Laboratory on Extreme-Scale Computing (JLESC) was initiated at the same time lossy compression for scientific data became an important topic for the scientific communities. The teams involved in the JLESC played and are still playing an important role in developing the research, techniques, methods, and technologies making lossy compression for scientific data a key tool for scientists and engineers. In this paper, we present the evolution of lossy compression for scientific data from 2015, describing the situation before the JLESC started, the evolution of this discipline in the past 8 years (until 2023) through the prism of the JLESC collaborations on this topic and some of the remaining open research questions.

引用

页数：27

共 50 条

[31] Parallel Tensor Compression for Large-Scale Scientific Data
Austin, Woody
Ballard, Grey
Kolda, Tamara G.
2016 IEEE 30TH INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM (IPDPS 2016), 2016, : 912 - 922
[32] Mesh data management in large-scale scientific computing
Chen, Hong
Zheng, Winmin
PROCEEDINGS OF THE THIRD CHINAGRID ANNUAL CONFERENCE, 2008, : 144 - 152
[33] WAVESZ: A Hardware-Algorithm Co-Design of Efficient Lossy Compression for Scientific Data
Tian, Jiannan
Di, Sheng
Zhang, Chengming
Liang, Xin
Jin, Sian
Cheng, Dazhao
Tao, Dingwen
Cappello, Franck
PROCEEDINGS OF THE 25TH ACM SIGPLAN SYMPOSIUM ON PRINCIPLES AND PRACTICE OF PARALLEL PROGRAMMING (PPOPP '20), 2020, : 74 - 88
[34] Extreme Data Science at the National Energy Research Scientific Computing (NERSC) Center
Dosanjh, Sudip
Canon, Shane
Deslippe, Jack
Fagnan, Kjiersten
Gerber, Richard
Gerhardt, Lisa
Hick, Jason
Jacobsen, Douglas
Skinner, David
Wright, Nicholas J.
PARALLEL COMPUTING: ACCELERATING COMPUTATIONAL SCIENCE AND ENGINEERING (CSE), 2014, 25 : 3 - 18
[35] Big data and extreme-scale computing: Pathways to Convergence-Toward a shaping strategy for a future software and data ecosystem for scientific inquiry
Asch, M.
Moore, T.
Badia, R.
Beck, M.
Beckman, P.
Bidot, T.
Bodin, F.
Cappello, F.
Choudhary, A.
de Supinski, B.
Deelman, E.
Dongarra, J.
Dubey, A.
Fox, G.
Fu, H.
Girona, S.
Gropp, W.
Heroux, M.
Ishikawa, Y.
Keahey, K.
Keyes, D.
Kramer, W.
Lavignon, J-F
Lu, Y.
Matsuoka, S.
Mohr, B.
Reed, D.
Requena, S.
Saltz, J.
Schulthess, T.
Stevens, R.
Swany, M.
Szalay, A.
Tang, W.
Varoquaux, G.
Vilotte, J-P
Wisniewski, R.
Xu, Z.
Zacharov, I.
INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS, 2018, 32 (04): : 435 - 479
[36] On a Storage System Software Stack for Extreme Scale Data Centric Computing
Narasimhamurthy, Sai
2017 46TH INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING WORKSHOPS (ICPPW), 2017, : 228 - 230
[37] Data Provenance Hybridization Supporting Extreme-Scale Scientific WorkflowApplications
Elsethagen, Todd
Stephan, Eric
Raju, Bibi
Schram, Malachi
MacDuff, Matt
Kerbyson, Darrern
van Dam, Kerstin Kleese
Singh, Alok
Altintas, Ilkay
2016 NEW YORK SCIENTIFIC DATA SUMMIT (NYSDS), 2016,
[38] FusionFS: Toward Supporting Data-Intensive Scientific Applications on Extreme-Scale High-Performance Computing Systems
Zhao, Dongfang
Zhang, Zhao
Zhou, Xiaobing
Li, Tonglin
Wang, Ke
Kimpe, Dries
Carns, Philip
Ross, Robert
Raicu, Ioan
2014 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2014, : 61 - 70
[39] cuSZ: An Efficient GPU-Based Error-Bounded Lossy Compression Framework for Scientific Data
Tian, Jiannan
Di, Sheng
Zhao, Kai
Rivera, Cody
Fulp, Megan Hickman
Underwood, Robert
Jin, Sian
Liang, Xin
Calhoun, Jon
Tao, Dingwen
Cappello, Franck
PACT '20: PROCEEDINGS OF THE ACM INTERNATIONAL CONFERENCE ON PARALLEL ARCHITECTURES AND COMPILATION TECHNIQUES, 2020, : 3 - 15
[40] Unbalanced Parallel I/O: An Often-Neglected Side Effect of Lossy Scientific Data Compression
Wang, Xinying
Wan, Lipeng
Chen, Jieyang
Gong, Qian
Whitney, Ben
Wang, Jinzhen
Gainaru, Ana
Liu, Qing
Podhorszki, Norbert
Zhao, Dongfang
Yan, Feng
Klasky, Scott
PROCEEDINGS OF THE 7TH INTERNATIONAL WORKSHOP ON DATA ANALYSIS AND REDUCTION FOR BIG SCIENTIFIC DATA (DRBSD-7), 2021, : 26 - 32

← 1 2 3 4 5 →