- Notifications
You must be signed in to change notification settings - Fork3
Fast deflate implementation with zopfli-like ratios and a streaming API
License
emmanuel-marty/zultra
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
zultra is a command-line tool and a compression-only library that produces compressed bitstreams in the zlib (RFC 1950), deflate (RFC 1951), and gzip (RFC 1952) formats, fully compatible with exisiting tools and libraries using these formats, such as zip and png.
The zultra library creates near-optimal compressed bitstreams with a ratio similar to zopfli (gaining around 4% on zlib), at approximately 25-50% of the speed of zlib compression level 9, on average.
zultra is written in plain C. The maximum block size (used to optimize the output) can be tuned, for instance for on-device compression scenarios. The compressor fully supports streaming with an API very similar to zlib.
Example benchmarks (using lzbench 1.7.3):
enwik8 (100000000)Compressor name Compress. Decompress. Compr. size Ratio Filenamememcpy 8846 MB/s 9006 MB/s 100000000 100.00 enwik8zultra 1.0.0 3.38 MB/s 269 MB/s 35029585 35.03 enwik8zlib 1.2.11 -9 16 MB/s 253 MB/s 36475792 36.48 enwik8libdeflate 1.0 -12 6.55 MB/s 581 MB/s 35100568 35.10 enwik8zopfli 1.0.2 0.38 MB/s 264 MB/s 34966066 34.97 enwik8mozilla, silesia corpus (51220480)Compressor name Compress. Decompress. Compr. size Ratio Filenamememcpy 9488 MB/s 9465 MB/s 51220480 100.00 mozillazultra 1.0.0 3.48 MB/s 277 MB/s 18280189 35.69 mozillazlib 1.2.11 -9 6.70 MB/s 279 MB/s 19044396 37.18 mozillalibdeflate 1.0 -12 7.65 MB/s 586 MB/s 18308548 35.74 mozillazopfli 1.0.2 0.28 MB/s 279 MB/s 18317347 35.76 mozillapariah.utx (24375895)Compressor name Compress. Decompress. Compr. size Ratio Filenamememcpy 9537 MB/s 9629 MB/s 24375895 100.00 pariah.utxzultra 1.0.0 3.75 MB/s 302 MB/s 7892356 32.38 pariah.utxzlib 1.2.11 -9 3.84 MB/s 301 MB/s 8214524 33.70 pariah.utxlibdeflate 1.0 -12 7.05 MB/s 675 MB/s 7914073 32.47 pariah.utxzopfli 1.0.2 0.16 MB/s 302 MB/s 7886453 32.35 pariah.utxbootstrap.min.js (48944)Compressor name Compress. Decompress. Compr. size Ratio Filenamememcpy 38813 MB/s 41025 MB/s 48944 100.00 bootstrap.min.jszultra 1.0.0 3.15 MB/s 353 MB/s 12599 25.74 bootstrap.min.jszlib 1.2.11 -9 32 MB/s 355 MB/s 13034 26.63 bootstrap.min.jslibdeflate 1.0 -12 5.92 MB/s 881 MB/s 12617 25.78 bootstrap.min.jszopfli 1.0.2 0.42 MB/s 345 MB/s 12599 25.74 bootstrap.min.jsInspirations:
- The original library itself,zlib by Jean-loup Gailly and Mark Adler.
- Some huffman RLE encoding optimizations tricks inZopfli by Lode Vandevenne and Jyrki Alakuijala.
- The suffix array intervals inWimlib by Eric Biggers.
- The block splitting heuristics inlibdeflate, also by Eric Biggers.
- Some general LZ-huffman ideas inlinzip2 by Gary Linscott.
- Fast huffman codelength building fromthis research paper by Moffat and Katajainen.
License:
- The zultra code is available under the Zlib license.
- The match finder (matchfinder.c) is available under the CC0 license due to using portions of code from Eric Bigger's Wimlib in the suffix array-based matchfinder.
- huffutils.c is available under the Apache License, Version 2.0.
About
Fast deflate implementation with zopfli-like ratios and a streaming API