Broth: A General-Purpose Data Compressor

被引:41
|
作者
Alakuijala, Jyrki [1 ,4 ]
Farruggia, Andrea [2 ,3 ]
Ferragina, Paolo [2 ,3 ]
Kliuchnikov, Eugene [1 ,4 ]
Obryk, Robert [1 ,4 ]
Szabadka, Zoltan [1 ,4 ]
Vandevenne, Lode [1 ,4 ]
机构
[1] Google Res, Zurich, Switzerland
[2] Univ Pisa, Pisa, Italy
[3] Dipartimento Informat, Largo B Pontecorvo 3, I-56127 Pisa, Italy
[4] Google, Brandschenkestr 110, CH-8002 Zurich, Switzerland
关键词
Data compression; Lempel-Ziv parsing; Treaps; NP-completeness; shortest paths; experiments;
D O I
10.1145/3231935
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Broth is an open source general-purpose data compressor introduced by Google in late 2013 and now adopted in most known browsers and Web servers. It is publicly available on GitHub and its data format was submitted as RFC 7932 in July 2016. Broth i is based on the Lempel-Ziv compression scheme and planned as a generic replacement of Gzip and ZLib. The main goal in its design was to compress data on the Internet, which meant optimizing the resources used at decoding time, while achieving maximal compression density. This article is intended to provide the first thorough, systematic description of the Brotli format as well as a detailed computational and experimental analysis of the main algorithmic blocks underlying the current encoder implementation, together with a comparison against compressors of different families constituting the state-of-the-art either in practice or in theory. This treatment will allow us to raise a set of new algorithmic and software engineering problems that deserve further attention from the scientific community.
引用
下载
收藏
页数:30
相关论文
共 50 条
  • [21] A Comparison of General-Purpose Distributed Systems for Data Processing
    Li, Jinfeng
    Cheng, James
    Zhao, Yunjian
    Yang, Fan
    Huang, Yuzhen
    Chen, Haipeng
    Zhao, Ruihao
    2016 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2016, : 378 - 383
  • [22] A GENERAL-PURPOSE PROGRAM FOR MULTIVARIATE DATA-ANALYSIS
    KVALHEIM, OM
    KARSTANG, TV
    CHEMOMETRICS AND INTELLIGENT LABORATORY SYSTEMS, 1987, 2 (1-3) : 235 - 237
  • [23] Conservative covariance for general-purpose nuclear data evaluation
    Tamagno, Pierre
    EUROPEAN PHYSICAL JOURNAL A, 2021, 57 (02):
  • [24] Finger trees: a simple general-purpose data structure
    Hinze, R
    Paterson, R
    JOURNAL OF FUNCTIONAL PROGRAMMING, 2006, 16 : 197 - 217
  • [26] GENERAL-PURPOSE FREE FORMAT INPUT DATA SYSTEM
    SURANA, KS
    COMPUTERS & STRUCTURES, 1979, 10 (04) : 583 - 597
  • [27] A general-purpose baseline estimation algorithm for spectroscopic data
    Barkauskas, Donald A.
    Rocke, David M.
    ANALYTICA CHIMICA ACTA, 2010, 657 (02) : 191 - 197
  • [28] Towards Free Data Selection with General-Purpose Models
    Xie, Yichen
    Ding, Mingyu
    Tomizuka, Masayoshi
    Zhan, Wei
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [29] GENERAL-PURPOSE SYSTEM FOR DATA ACQUISITION AND INSTRUMENT CONTROL
    EHRMANN, CH
    ENGEL, AE
    IEEE TRANSACTIONS ON NUCLEAR SCIENCE, 1979, 26 (04) : 4459 - 4467
  • [30] Extracting general-purpose features from LIDAR data
    Li, Yangming
    Olson, Edwin B.
    2010 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2010, : 1388 - 1393