Broth: A General-Purpose Data Compressor

被引:41
|
作者
Alakuijala, Jyrki [1 ,4 ]
Farruggia, Andrea [2 ,3 ]
Ferragina, Paolo [2 ,3 ]
Kliuchnikov, Eugene [1 ,4 ]
Obryk, Robert [1 ,4 ]
Szabadka, Zoltan [1 ,4 ]
Vandevenne, Lode [1 ,4 ]
机构
[1] Google Res, Zurich, Switzerland
[2] Univ Pisa, Pisa, Italy
[3] Dipartimento Informat, Largo B Pontecorvo 3, I-56127 Pisa, Italy
[4] Google, Brandschenkestr 110, CH-8002 Zurich, Switzerland
关键词
Data compression; Lempel-Ziv parsing; Treaps; NP-completeness; shortest paths; experiments;
D O I
10.1145/3231935
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Broth is an open source general-purpose data compressor introduced by Google in late 2013 and now adopted in most known browsers and Web servers. It is publicly available on GitHub and its data format was submitted as RFC 7932 in July 2016. Broth i is based on the Lempel-Ziv compression scheme and planned as a generic replacement of Gzip and ZLib. The main goal in its design was to compress data on the Internet, which meant optimizing the resources used at decoding time, while achieving maximal compression density. This article is intended to provide the first thorough, systematic description of the Brotli format as well as a detailed computational and experimental analysis of the main algorithmic blocks underlying the current encoder implementation, together with a comparison against compressors of different families constituting the state-of-the-art either in practice or in theory. This treatment will allow us to raise a set of new algorithmic and software engineering problems that deserve further attention from the scientific community.
引用
下载
收藏
页数:30
相关论文
共 50 条
  • [1] GENERAL-PURPOSE DATA-ANALYSIS
    HOLEWINSKI, PK
    LABORATORY MEDICINE, 1987, 18 (07) : 475 - 475
  • [2] GENERAL-PURPOSE DATA EXTRACTION LANGUAGE
    TSUI, WH
    GELDER, PV
    BEHAVIOR RESEARCH METHODS & INSTRUMENTATION, 1979, 11 (02): : 199 - 204
  • [3] A GENERAL-PURPOSE DATA-ENTRY PROGRAM
    JACKY, JP
    KALET, IJ
    COMMUNICATIONS OF THE ACM, 1983, 26 (06) : 409 - 417
  • [4] TRACE: A Fast Transformer-based General-Purpose Lossless Compressor
    Mao, Yu
    Cui, Yufei
    Kuo, Tei-Wei
    Xue, Chun Jason
    PROCEEDINGS OF THE ACM WEB CONFERENCE 2022 (WWW'22), 2022, : 1829 - 1838
  • [5] A parallel general-purpose synthetic data generator
    Hoag, Joseph E.
    Thompson, Craig W.
    SIGMOD RECORD, 2007, 36 (01) : 19 - 24
  • [6] DATA VISUALIZATION USING A GENERAL-PURPOSE RENDERER
    DOI, A
    AONO, M
    URANO, N
    SUGIMOTO, K
    IBM JOURNAL OF RESEARCH AND DEVELOPMENT, 1991, 35 (1-2) : 45 - 58
  • [7] Data visualization using a general-purpose renderer
    Doi, A., 1600, (35): : 1 - 2
  • [8] ENDAT - A GENERAL-PURPOSE DATA ACQUISITION TRIGGER
    FINOCCHIARO, P
    ANZALONE, A
    MAIOLINO, C
    NUCLEAR INSTRUMENTS & METHODS IN PHYSICS RESEARCH SECTION A-ACCELERATORS SPECTROMETERS DETECTORS AND ASSOCIATED EQUIPMENT, 1988, 271 (03): : 563 - 566
  • [9] A GENERAL-PURPOSE DATA ACQUISITION-SYSTEM
    GEHMAN, LP
    WORTMAN, RS
    BROWN, BD
    SHACKELFORD, JF
    AMERICAN CERAMIC SOCIETY BULLETIN, 1984, 63 (08): : 1012 - 1012
  • [10] GENERAL-PURPOSE UTILITY FOR DATA ACQUISITION AND CONTROL
    MOORE, CF
    INSTRUMENTS & CONTROL SYSTEMS, 1972, 45 (07): : 15 - &