- Notifications
You must be signed in to change notification settings - Fork31
Pure Nim implementation of deflate, zlib, gzip and zip.
License
guzba/zippy
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
nimble install zippy
Zippy is an implementation ofDEFLATE,ZLIB andGZIP data compression formats.
Zippy can also openZIP archives (.zip) andTarballs (.tar, .tar.gz, .tgz, .taz).
The goal of this library is to be a pure Nim implementation that is small, performant and dependency-free.
To ensure Zippy is compatible with other implementations,tests/validate.nim can be run. This script verifies that data compressed by Zippy can be uncompressed by other implementations (and that other implementations can uncompress data compressed by Zippy).
This library works well using Nim's--gc:arc and--gc:orc as well as the default garbage collector. This library also works using bothnim c andnim cpp, in addition to--cc:vcc on Windows.
Simple examples using Zippy can be found in theexamples/ folder.
- HTTP client gzip
- HTTP server gzip
- Extract individual files fromzip archive.
- Extract everything from azip archive ortarball
Benchmarks can be run comparing different deflate implementations. My benchmarking shows this library performs very well, around 1.5x - 2x faster than zlib found on a fresh Linux install. Check the performance yourself by runningtests/benchmark.nim.
nim c --gc:arc -d:release -r .\tests\benchmark.nim
The times below are measured on a Ryzen 5 5600X.
https://github.com/guzba/zippy uncompressname ............................... min time avg time std dv runsalice29.txt.gz ..................... 0.233 ms 0.235 ms ±0.003 x1000urls.10K.gz ........................ 1.140 ms 1.148 ms ±0.007 x1000rfctest3.gz ........................ 0.047 ms 0.048 ms ±0.001 x1000randtest3.gz ....................... 0.001 ms 0.001 ms ±0.000 x1000paper-100k.pdf.gz .................. 0.210 ms 0.212 ms ±0.001 x1000geo.protodata.gz ................... 0.068 ms 0.071 ms ±0.002 x1000tor-list.gz ....................... 27.297 ms 27.520 ms ±0.277 x182https://github.com/nim-lang/zip uncompressalice29.txt.gz ..................... 0.397 ms 0.403 ms ±0.004 x1000urls.10K.gz ........................ 1.719 ms 1.731 ms ±0.009 x1000rfctest3.gz ........................ 0.054 ms 0.055 ms ±0.002 x1000randtest3.gz ....................... 0.008 ms 0.008 ms ±0.000 x1000paper-100k.pdf.gz .................. 0.250 ms 0.252 ms ±0.001 x1000geo.protodata.gz ................... 0.126 ms 0.132 ms ±0.005 x1000tor-list.gz ....................... 36.613 ms 37.061 ms ±0.423 x135https://github.com/guzba/zippy compress [best speed]name ............................... min time avg time std dv runsalice29.txt ........................ 0.643 ms 0.655 ms ±0.017 x1000urls.10K ........................... 1.943 ms 1.959 ms ±0.022 x1000rfctest3.gold ...................... 0.119 ms 0.121 ms ±0.003 x1000randtest3.gold ..................... 0.005 ms 0.006 ms ±0.001 x1000paper-100k.pdf ..................... 0.230 ms 0.235 ms ±0.003 x1000geo.protodata ...................... 0.192 ms 0.195 ms ±0.003 x1000gzipfiletest.txt ................... 0.002 ms 0.002 ms ±0.000 x1000tor-list.gold ..................... 26.106 ms 26.418 ms ±0.370 x189https://github.com/nim-lang/zip compress [best speed]alice29.txt ........................ 1.236 ms 1.245 ms ±0.010 x1000urls.10K ........................... 5.155 ms 5.222 ms ±0.092 x952rfctest3.gold ...................... 0.205 ms 0.232 ms ±0.009 x1000randtest3.gold ..................... 0.076 ms 0.097 ms ±0.013 x1000paper-100k.pdf ..................... 1.250 ms 1.276 ms ±0.023 x1000geo.protodata ...................... 0.313 ms 0.320 ms ±0.006 x1000gzipfiletest.txt ................... 0.006 ms 0.008 ms ±0.001 x1000tor-list.gold .................... 178.197 ms 179.970 ms ±1.559 x28https://github.com/guzba/zippy compress [default]name ............................... min time avg time std dv runsalice29.txt ........................ 2.361 ms 2.379 ms ±0.024 x1000urls.10K .......................... 13.364 ms 13.432 ms ±0.036 x372rfctest3.gold ...................... 0.335 ms 0.342 ms ±0.009 x1000randtest3.gold ..................... 0.048 ms 0.049 ms ±0.000 x1000paper-100k.pdf ..................... 0.831 ms 0.843 ms ±0.010 x1000geo.protodata ...................... 0.563 ms 0.570 ms ±0.007 x1000gzipfiletest.txt ................... 0.008 ms 0.008 ms ±0.001 x1000tor-list.gold .................... 409.542 ms 411.858 ms ±1.779 x13https://github.com/nim-lang/zip compress [default]alice29.txt ........................ 5.726 ms 5.766 ms ±0.053 x862urls.10K .......................... 13.049 ms 13.106 ms ±0.057 x381rfctest3.gold ...................... 0.637 ms 0.644 ms ±0.007 x1000randtest3.gold ..................... 0.083 ms 0.087 ms ±0.007 x1000paper-100k.pdf ..................... 1.467 ms 1.490 ms ±0.023 x1000geo.protodata ...................... 0.867 ms 0.879 ms ±0.011 x1000gzipfiletest.txt ................... 0.009 ms 0.010 ms ±0.001 x1000tor-list.gold .................... 244.424 ms 246.601 ms ±1.630 x21nimble test
To prevent Zippy from causing a crash or otherwise misbehaving on bad input data, a fuzzer has been run against it. You can run the fuzzer any time by runningnim c -r tests/fuzz.nim andnim c -r tests/stress.nim
About
Pure Nim implementation of deflate, zlib, gzip and zip.