Lossy Scientific Data Compression With SPERR

被引：7
|
作者：
Li, Shaomeng [1 ]
Lindstrom, Peter [2 ]
Clyne, John [1 ]
机构：
[1] Natl Ctr Atmospher Res, Boulder, CO 80305 USA
[2] Lawrence Livermore Natl Lab, Lawrence, KS USA
来源：
2023 IEEE INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM, IPDPS | 2023年
基金：
美国国家科学基金会;
关键词：
MULTILEVEL TECHNIQUES; IMAGE COMPRESSION; EFFICIENT; REDUCTION; QUANTIZATION;
D O I：
10.1109/IPDPS54959.2023.00104
中图分类号：
TP3 [计算技术、计算机技术];
学科分类号：
0812 ;
摘要：
As the need for data reduction in high-performance computing (HPC) continues to grow, we introduce a new and highly effective tool to help achieve this goal-SPERR. SPERR is a versatile lossy compressor for structured scientific data; it is built on top of an advanced wavelet compression algorithm, SPECK, and provides additional capabilities valued in HPC environments. These capabilities include parallel execution for large volumes and a compression mode that satisfies a maximum point-wise error tolerance. Evaluation shows that in most settings SPERR achieves the best rate-distortion trade-off among current popular lossy scientific data compressors.
引用
收藏
页码：1007 / 1017
页数：11
相关论文
共 50 条
  • [1] Understanding and Modeling Lossy Compression Schemes on HPC Scientific Data
    Lu, Tao
    Liu, Qing
    He, Xubin
    Luo, Huizhang
    Suchyta, Eric
    Choi, Jong
    Podhorszki, Norbert
    Klasky, Scott
    Wolf, Mathew
    Liu, Tong
    Qiao, Zhenbo
    2018 32ND IEEE INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM (IPDPS), 2018, : 348 - 357
  • [2] Improving Performance of Data Dumping with Lossy Compression for Scientific Simulation
    Liang, Xin
    Di, Sheng
    Tao, Dingwen
    Li, Sihuan
    Nicolae, Bogdan
    Chen, Zizhong
    Cappello, Franck
    2019 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER), 2019, : 340 - 350
  • [3] Pointwise redundancy in lossy data compression and universal lossy data compression
    Kontoyiannis, I
    IEEE TRANSACTIONS ON INFORMATION THEORY, 2000, 46 (01) : 136 - 152
  • [4] Lossy compression of scientific spacecraft data using wavelets. Application to the CASSINI spacecraft data compression
    Belmon, L.
    Benoit-Cattin, Hugues
    Baskurt, A.
    Bougeret, J.-L.
    1600, EDP Sciences (386):
  • [5] Lossy compression of scientific spacecraft data using wavelets. Application to the CASSINI spacecraft data compression
    Belmon, L
    Benoit-Cattin, H
    Baskurt, A
    Bougeret, JL
    ASTRONOMY & ASTROPHYSICS, 2002, 386 (03): : 1143 - 1152
  • [6] Lossy compression of scientific simulation data: from visualization to checkpoint/restart
    Cappello, Franck
    2018 IEEE INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM WORKSHOPS (IPDPSW 2018), 2018, : 1205 - 1205
  • [7] DPZ: Improving Lossy Compression Ratio with Information Retrieval on Scientific Data
    Zhang, Jialing
    Chen, Jiaxi
    Zhuo, Xiaoyan
    Moon, Aekyeung
    Son, Seung Woo
    2021 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER 2021), 2021, : 320 - 331
  • [8] Z-checker: A framework for assessing lossy compression of scientific data
    Tao, Dingwen
    Di, Sheng
    Guo, Hanqi
    Chen, Zizhong
    Cappello, Franck
    INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS, 2019, 33 (02): : 285 - 303
  • [9] Optimizing Error-Bounded Lossy Compression for Scientific Data on GPUs
    Tian, Jiannan
    Di, Sheng
    Yu, Xiaodong
    Rivera, Cody
    Zhao, Kai
    Jin, Sian
    Feng, Yunhe
    Liang, Xin
    Tao, Dingwen
    Cappello, Franck
    2021 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER 2021), 2021, : 283 - 293
  • [10] Fixed-PSNR Lossy Compression for Scientific Data Short paper
    Tao, Dingwen
    Di, Sheng
    Liang, Xin
    Chen, Zizhong
    Cappello, Franck
    2018 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING (CLUSTER), 2018, : 314 - 318
12345