• July 1st, 2023 -v1.16.7

  • zstd: Fix default level first dictionary encode#829
  • s2: add GetBufferCapacity() method by @GiedriusS in#832

• June 13, 2023 -v1.16.6

  • zstd: correctly ignore WithEncoderPadding(1) by @ianlancetaylor in#806
  • zstd: Add amd64 match length assembly#824
  • gzhttp: Handle informational headers by @rtribotte in#815
  • s2: Improve Better compression slightly#663

• Apr 16, 2023 -v1.16.5

  • zstd: readByte needs to use io.ReadFull by @jnoxon in#802
  • gzip: Fix WriterTo after initial read#804

• Apr 5, 2023 -v1.16.4

  • zstd: Improve zstd best efficiency by @greatroar and @klauspost in#784
  • zstd: Respect WithAllLitEntropyCompression#792
  • zstd: Fix amd64 not always detecting corrupt data#785
  • zstd: Various minor improvements by @greatroar in#788#794#795
  • s2: Fix huge block overflow#779
  • s2: Allow CustomEncoder fallback#780
  • gzhttp: Support ResponseWriter Unwrap() in gzhttp handler by @jgimenez in#799

• Mar 13, 2023 -v1.16.1

  • zstd: Speed up + improve best encoder by @greatroar in#776
  • gzhttp: Add optionalBREACH mitigation.#762#768#769#770#767
  • s2: Add Intel LZ4s converter#766
  • zstd: Minor bug fixes#771#772#773
  • huff0: Speed up compress1xDo by @greatroar in#774

• Feb 26, 2023 -v1.16.0

  • s2: AddDictionary support.#685
  • s2: Add Compression Size Estimate.#752
  • s2: Add support for custom stream encoder.#755
  • s2: Add LZ4 block converter.#748
  • s2: Support io.ReaderAt in ReadSeeker.#747
  • s2c/s2sx: Use concurrent decoding.#746

• Jan 21st, 2023 (v1.15.15)

  • deflate: Improve level 7-9#739
  • zstd: Add delta encoding support by @greatroar in#728
  • zstd: Various speed improvements by @greatroar#741#734#736#744#743#745
  • gzhttp: Add SuffixETag() and DropETag() options to prevent ETag collisions on compressed responses by @willbicks in#740

• Jan 3rd, 2023 (v1.15.14)

  • flate: Improve speed in big stateless blocks#718
  • zstd: Minor speed tweaks by @greatroar in#716#720
  • export NoGzipResponseWriter for custom ResponseWriter wrappers by @harshavardhana in#722
  • s2: Add example for indexing and existing stream#723

• Dec 11, 2022 (v1.15.13)

  • zstd: AddMaxEncodedSize to encoder#691
  • zstd: Various tweaks and improvements#693#695#696#701#702#703#704#705#706#707#708

• Oct 26, 2022 (v1.15.12)

  • zstd: Tweak decoder allocs.#680
  • gzhttp: Always deleteHeaderNoCompression#683

• Sept 26, 2022 (v1.15.11)

  • flate: Improve level 1-3 compression#678
  • zstd: Improve "best" compression by @nightwolfz in#677
  • zstd: Fix+reduce decompression allocations#668
  • zstd: Fix non-effective noescape tag#667

• Sept 16, 2022 (v1.15.10)

  • zstd: AddWithDecodeAllCapLimit#649
  • Add Go 1.19 - deprecate Go 1.16#651
  • flate: Improve level 5+6 compression#656
  • zstd: Improve "better" compression#657
  • s2: Improve "best" compression#658
  • s2: Improve "better" compression.#635
  • s2: Slightly faster non-assembly decompression#646
  • Use arrays for constant size copies#659

• July 21, 2022 (v1.15.9)

  • zstd: Fix decoder crash on amd64 (no BMI) on invalid input#645
  • zstd: Disable decoder extended memory copies (amd64) due to possible crashes#644
  • zstd: Allow single segments up to "max decoded size"#643

• July 13, 2022 (v1.15.8)

  • gzip: fix stack exhaustion bug in Reader.Read#641
  • s2: Add Index header trim/restore#638
  • zstd: Optimize seqdeq amd64 asm by @greatroar in#636
  • zstd: Improve decoder memcopy#637
  • huff0: Pass a single bitReader pointer to asm by @greatroar in#634
  • zstd: Branchless getBits for amd64 w/o BMI2 by @greatroar in#640
  • gzhttp: Remove header before writing#639

• June 29, 2022 (v1.15.7)

  • s2: Fix absolute forward seeks#633
  • zip: Merge upstream#631
  • zip: Re-add zip64 fix#624
  • zstd: translate fseDecoder.buildDtable into asm by @WojciechMula in#598
  • flate: Faster histograms#620
  • deflate: Use compound hcode#622

• June 3, 2022 (v1.15.6)

  • s2: Improve coding for long, close matches#613
  • s2c: Add Snappy/S2 stream recompression#611
  • zstd: Always use configured block size#605
  • zstd: Fix incorrect hash table placement for dict encoding in default#606
  • zstd: Apply default config to ZipDecompressor without options#608
  • gzhttp: Exclude more common archive formats#612
  • s2: Add ReaderIgnoreCRC#609
  • s2: Remove sanity load on index creation#607
  • snappy: Use dedicated function for scoring#614
  • s2c+s2d: Use official snappy framed extension#610

• May 25, 2022 (v1.15.5)

  • s2: Add concurrent stream decompression#602
  • s2: Fix final emit oob read crash on amd64#601
  • huff0: asm implementation of Decompress1X by @WojciechMula#596
  • zstd: Use 1 less goroutine for stream decoding#588
  • zstd: Copy literal in 16 byte blocks when possible#592
  • zstd: Speed up when WithDecoderLowmem(false)#599
  • zstd: faster next state update in BMI2 version of decode by @WojciechMula in#593
  • huff0: Do not check max size when reading table.#586
  • flate: Inplace hashing for level 7-9#590

• May 11, 2022 (v1.15.4)

  • huff0: decompress directly into output by @WojciechMula in#577
  • inflate: Keep dict on stack#581
  • zstd: Faster decoding memcopy in asm#583
  • zstd: Fix ignored crc#580

• May 5, 2022 (v1.15.3)

  • zstd: Allow to ignore checksum checking by @WojciechMula#572
  • s2: Fix incorrect seek for io.SeekEnd in#575

• Apr 26, 2022 (v1.15.2)

  • zstd: Add x86-64 assembly for decompression on streams and blocks. Contributed by@WojciechMula. Typically 2x faster.#528#531#545#537
  • zstd: Add options to ZipDecompressor and fixes#539
  • s2: Use sorted search for index#555
  • Minimum version is Go 1.16, added CI test on 1.18.

• Mar 11, 2022 (v1.15.1)

  • huff0: Add x86 assembly of Decode4X by @WojciechMula in#512
  • zstd: Reuse zip decoders in#514
  • zstd: Detect extra block data and report as corrupted in#520
  • zstd: Handle zero sized frame content size stricter in#521
  • zstd: Add stricter block size checks in#523

• Mar 3, 2022 (v1.15.0)

  • zstd: Refactor decoder#498
  • zstd: Add stream encoding without goroutines#505
  • huff0: Prevent single blocks exceeding 16 bits by @klauspost in#507
  • flate: Inline literal emission#509
  • gzhttp: Add zstd to transport#400
  • gzhttp: Make content-type optional#510

Both compression and decompression now supports "synchronous" stream operations. This means that whenever "concurrency" is set to 1, they will operate without spawning goroutines.

Stream decompression is now faster on asynchronous, since the goroutine allocation much more effectively splits the workload. On typical streams this will typically use 2 cores fully for decompression. When a stream has finished decoding no goroutines will be left over, so decoders can now safely be pooled and still be garbage collected.

While the release has been extensively tested, it is recommended to testing when upgrading.

• Feb 22, 2022 (v1.14.4)

  • flate: Fix rare huffman only (-2) corruption.#503
  • zip: Update deprecated CreateHeaderRaw to correctly call CreateRaw by @saracen in#502
  • zip: don't read data descriptor early by @saracen in#501 #501
  • huff0: Use static decompression buffer up to 30% faster#499#500

• Feb 17, 2022 (v1.14.3)

  • flate: Improve fastest levels compression speed ~10% more throughput.#482#489#490#491#494#478
  • flate: Faster decompression speed, ~5-10%.#483
  • s2: Faster compression with Go v1.18 and amd64 microarch level 3+.#484#486

• Jan 25, 2022 (v1.14.2)

  • zstd: improve header decoder by @dsnet#476
  • zstd: Add bigger default blocks#469
  • zstd: Remove unused decompression buffer#470
  • zstd: Fix logically dead code by @ningmingxiao#472
  • flate: Improve level 7-9#471#473
  • zstd: Add noasm tag for xxhash#475

• Jan 11, 2022 (v1.14.1)

  • s2: Add stream index in#462
  • flate: Speed and efficiency improvements in#439#461#455#452#458
  • zstd: Performance improvement in#420#456#437#467#468
  • zstd: add arm64 xxhash assembly in#464
  • Add garbled for binaries for s2 in#445

• Aug 30, 2021 (v1.13.5)

  • gz/zlib/flate: Alias stdlib errors#425
  • s2: Add block support to commandline tools#413
  • zstd: pooledZipWriter should return Writers to the same pool#426
  • Removed golang/snappy as external dependency for tests#421

• Aug 12, 2021 (v1.13.4)

  • Addsnappy replacement package.
  • zstd: Fix incorrect encoding in "best" mode#415

• Aug 3, 2021 (v1.13.3)

  • zstd: Improve Best compression#404
  • zstd: Fix WriteTo error forwarding#411
  • gzhttp: Return http.HandlerFunc instead of http.Handler. Unlikely breaking change.#406
  • s2sx: Fix max size error#399
  • zstd: Add optional stream content size on reset#401
  • zstd: use SpeedBestCompression for level >= 10#410

• Jun 14, 2021 (v1.13.1)

  • s2: Add full Snappy output support#396
  • zstd: Add configurableDecoder window size#394
  • gzhttp: Add header to skip compression#389
  • s2: Improve speed with bigger output margin#395

• Jun 3, 2021 (v1.13.0)

  • Addedgzhttp which allows wrapping HTTP servers and clients with GZIP compressors.
  • zstd: Detect short invalid signatures#382
  • zstd: Spawn decoder goroutine only if needed.#380

• May 25, 2021 (v1.12.3)

  • deflate: Better/faster Huffman encoding#374
  • deflate: Allocate less for history.#375
  • zstd: Forward read errors#373

• Apr 27, 2021 (v1.12.2)

  • zstd: Improve better/best compression#360#364#365
  • zstd: Add helpers to compress/decompress zstd inside zip files#363
  • deflate: Improve level 5+6 compression#367
  • s2: Improve better/best compression#358#359
  • s2: Load after checking src limit on amd64.#362
  • s2sx: Limit max executable size#368

• Apr 14, 2021 (v1.12.1)

  • snappy package removed. Upstream added as dependency.
  • s2: Better compression in "best" mode#353
  • s2sx: Add stdin input and detect pre-compressed from signature#352
  • s2c/s2d: Add http as possible input#348
  • s2c/s2d/s2sx: Always truncate when writing files#352
  • zstd: Reduce memory usage further when usingWithLowerEncoderMem#346
  • s2: Fix potential problem with amd64 assembly and profilers#349

• Mar 26, 2021 (v1.11.13)

  • zstd: Big speedup on small dictionary encodes#344#345
  • zstd: AddWithLowerEncoderMem encoder option#336
  • deflate: Improve entropy compression#338
  • s2: Clean up and minor performance improvement in best#341

• Mar 5, 2021 (v1.11.12)

  • s2: Adds2sx binary that createsself extracting archives.
  • s2: Speed up decompression on non-assembly platforms#328

• Mar 1, 2021 (v1.11.9)

  • s2: Add ARM64 decompression assembly. Around 2x output speed.#324
  • s2: Improve "better" speed and efficiency.#325
  • s2: Fix binaries.

• Feb 25, 2021 (v1.11.8)

  • s2: Fixed occasional out-of-bounds write on amd64. Upgrade recommended.
  • s2: Add AMD64 assembly for better mode. 25-50% faster.#315
  • s2: Less upfront decoder allocation.#322
  • zstd: Faster "compression" of incompressible data.#314
  • zip: Fix zip64 headers.#313

• Jan 14, 2021 (v1.11.7)

  • Use Bytes() interface to get bytes across packages.#309
  • s2: Add 'best' compression option.#310
  • s2: Add ReaderMaxBlockSize, changess2.NewReader signature to include varargs.#311
  • s2: Fix crash on small better buffers.#308
  • s2: Clean up decoder.#312

• Jan 7, 2021 (v1.11.6)

  • zstd: Make decoder allocations smaller#306
  • zstd: Free Decoder resources when Reset is called with a nil io.Reader#305

• Dec 20, 2020 (v1.11.4)

  • zstd: Add Best compression mode#304
  • Add header decoder#299
  • s2: Add uncompressed stream option#297
  • Simplify/speed up small blocks with known max size.#300
  • zstd: Always reset literal dict encoder#303

• Nov 15, 2020 (v1.11.3)

  • inflate: 10-15% faster decompression#293
  • zstd: Tweak DecodeAll default allocation#295

• Oct 11, 2020 (v1.11.2)

  • s2: Fix out of bounds read in "better" block compression#291

• Oct 1, 2020 (v1.11.1)

  • zstd: Set allLitEntropy true in default configuration#286

• Sept 8, 2020 (v1.11.0)

  • zstd: Add experimental compressiondictionaries#281
  • zstd: Fix mixed Write and ReadFrom calls#282
  • inflate/gz: Limit variable shifts, ~5% faster decompression#274

• July 8, 2020 (v1.10.11)

  • zstd: Fix extra block when compressing with ReadFrom.#278
  • huff0: Also populate compression table when reading decoding table.#275

• June 23, 2020 (v1.10.10)

  • zstd: Skip entropy compression in fastest mode when no matches.#270

• June 16, 2020 (v1.10.9):

  • zstd: API change for specifying dictionaries. See#268
  • zip: update CreateHeaderRaw to handle zip64 fields.#266
  • Fuzzit tests removed. The service has been purchased and is no longer available.

• June 5, 2020 (v1.10.8):

  • 1.15x faster zstd block decompression.#265

• June 1, 2020 (v1.10.7):

  • Added zstd decompressiondictionary support
  • Increase zstd decompression speed up to 1.19x.#259
  • Remove internal reset call in zstd compression and reduce allocations.#263

• May 21, 2020: (v1.10.6)

  • zstd: Reduce allocations while decoding.#258,#252
  • zstd: Stricter decompression checks.

• April 12, 2020: (v1.10.5)

  • s2-commands: Flush output when receiving SIGINT.#239

• Apr 8, 2020: (v1.10.4)

  • zstd: Minor/special case optimizations.#251,#250,#249,#247

• Mar 11, 2020: (v1.10.3)

  • s2: Use S2 encoder in pure Go mode for Snappy output as well.#245
  • s2: Fix pure Go block encoder.#244
  • zstd: Added "better compression" mode.#240
  • zstd: Improve speed of fastest compression mode by 5-10%#241
  • zstd: Skip creating encoders when not needed.#238

• Feb 27, 2020: (v1.10.2)

  • Close to 50% speedup in inflate (gzip/zip decompression).#236#234#232
  • Reduce deflate level 1-6 memory usage up to 59%.#227

• Feb 18, 2020: (v1.10.1)

  • Fix zstd crash when resetting multiple times without sending data.#226
  • deflate: Fix dictionary use on level 1-6.#224
  • Remove deflate writer reference when closing.#224

• Feb 4, 2020: (v1.10.0)

  • Add optional dictionary tostateless deflate. Breaking change, sendnil for previous behaviour.#216
  • Fix buffer overflow on repeated small block deflate.#218
  • Allow copying content from an existing ZIP file without decompressing+compressing.#214
  • AddedS2 AMD64 assembler and various optimizations. Stream speed >10GB/s.#186

• Jan 20,2020 (v1.9.8) Optimize gzip/deflate with better size estimates and faster table generation.#207 byluyu6056,#206.
• Jan 11, 2020: S2 Encode/Decode will use provided buffer if capacity is big enough.#204
• Jan 5, 2020: (v1.9.7) Fix another zstd regression in v1.9.5 - v1.9.6 removed.
• Jan 4, 2020: (v1.9.6) Regression in v1.9.5 fixed causing corrupt zstd encodes in rare cases.
• Jan 4, 2020: Faster IO ins2c + s2d commandline tools compression/decompression.#192
• Dec 29, 2019: Removed v1.9.5 since fuzz tests showed a compatibility problem with the reference zstandard decoder.
• Dec 29, 2019: (v1.9.5) zstd: 10-20% faster block compression.#199
• Dec 29, 2019:zip package updated with latest Go features
• Dec 29, 2019: zstd: Single segment flag condintions tweaked.#197
• Dec 18, 2019: s2: Faster compression when ReadFrom is used.#198
• Dec 10, 2019: s2: Fix repeat length output when just above at 16MB limit.
• Dec 10, 2019: zstd: Add function to get decoder as io.ReadCloser.#191
• Dec 3, 2019: (v1.9.4) S2: limit max repeat length.#188
• Dec 3, 2019: AddWithNoEntropyCompression to zstd#187
• Dec 3, 2019: Reduce memory use for tests. Check for leaked goroutines.
• Nov 28, 2019 (v1.9.3) Less allocations in stateless deflate.
• Nov 28, 2019: 5-20% Faster huff0 decode. Impacts zstd as well.#184
• Nov 12, 2019 (v1.9.2) AddedStateless Compression for gzip/deflate.
• Nov 12, 2019: Fixed zstd decompression of large single blocks.#180
• Nov 11, 2019: Set defaults2c block size to 4MB.
• Nov 11, 2019: Reduce inflate memory use by 1KB.
• Nov 10, 2019: Less allocations in deflate bit writer.
• Nov 10, 2019: Fix inconsistent error returned by zstd decoder.
• Oct 28, 2019 (v1.9.1) ztsd: Fix crash when compressing blocks.#174
• Oct 24, 2019 (v1.9.0) zstd: Fix rare data corruption#173
• Oct 24, 2019 zstd: Fix huff0 out of buffer write#171 and always return errors#172
• Oct 10, 2019: Big deflate rewrite, 30-40% faster with better compression#105

• Oct 10, 2019: (v1.8.6) zstd: Allow partial reads to get flushed data.#169
• Oct 3, 2019: Fix inconsistent results on broken zstd streams.
• Sep 25, 2019: Added-rm (remove source files) and-q (no output except errors) tos2c ands2dcommands
• Sep 16, 2019: (v1.8.4) Adds2c ands2dcommandline tools.
• Sep 10, 2019: (v1.8.3) Fix s2 decoderSkip.
• Sep 7, 2019: zstd: AddedWithWindowSize, contributed byianwilkes.
• Sep 5, 2019: (v1.8.2) AddWithZeroFrames which adds full zero payload block encoding option.
• Sep 5, 2019: Lazy initialization of zstandard predefined en/decoder tables.
• Aug 26, 2019: (v1.8.1) S2: 1-2% compression increase in "better" compression mode.
• Aug 26, 2019: zstd: Check maximum size of Huffman 1X compressed literals while decoding.
• Aug 24, 2019: (v1.8.0) AddedS2 compression, a high performance replacement for Snappy.
• Aug 21, 2019: (v1.7.6) Fixed minor issues found by fuzzer. One could lead to zstd not decompressing.
• Aug 18, 2019: Addfuzzit continuous fuzzing.
• Aug 14, 2019: zstd: Skip incompressible data 2x faster.#147
• Aug 4, 2019 (v1.7.5): Better literal compression.#146
• Aug 4, 2019: Faster zstd compression.#143#144
• Aug 4, 2019: Faster zstd decompression.#145#143#142
• July 15, 2019 (v1.7.4): Fix double EOF block in rare cases on zstd encoder.
• July 15, 2019 (v1.7.3): Minor speedup/compression increase in default zstd encoder.
• July 14, 2019: zstd decoder: Fix decompression error on multiple uses with mixed content.
• July 7, 2019 (v1.7.2): Snappy update, zstd decoder potential race fix.
• June 17, 2019: zstd decompression bugfix.
• June 17, 2019: fix 32 bit builds.
• June 17, 2019: Easier use in modules (less dependencies).
• June 9, 2019: New stronger "default"zstd compression mode. Matches zstd default compression ratio.
• June 5, 2019: 20-40% throughput inzstandard compression and better compression.
• June 5, 2019: deflate/gzip compression: Reduce memory usage of lower compression levels.
• June 2, 2019: Addedzstandard compression!
• May 25, 2019: deflate/gzip: 10% faster bit writer, mostly visible in lower levels.
• Apr 22, 2019:zstd decompression added.
• Aug 1, 2018: Addedhuff0 README.
• Jul 8, 2018: AddedPerformance Update 2018 below.
• Jun 23, 2018: MergedGo 1.11 inflate optimizations. Go 1.9 is now required. Backwards compatible version tagged withv1.3.0.
• Apr 2, 2018: Addedhuff0 en/decoder. Experimental for now, API may change.
• Mar 4, 2018: AddedFSE Entropy en/decoder. Experimental for now, API may change.
• Nov 3, 2017: Add compressionEstimate function.
• May 28, 2017: Reduce allocations when resetting decoder.
• Apr 02, 2017: Change back to official crc32, since changes were merged in Go 1.7.
• Jan 14, 2017: Reduce stack pressure due to array copies. SeeIssue #18625.
• Oct 25, 2016: Level 2-4 have been rewritten and now offers significantly better performance than before.
• Oct 20, 2016: Port zlib changes from Go 1.7 to fix zlib writer issue. Please update.
• Oct 16, 2016: Go 1.7 changes merged. Apples to apples this package is a few percent faster, but has a significantly better balance between speed and compression per level.
• Mar 24, 2016: Always attempt Huffman encoding on level 4-7. This improves base 64 encoded data compression.
• Mar 24, 2016: Small speedup for level 1-3.
• Feb 19, 2016: Faster bit writer, level -2 is 15% faster, level 1 is 4% faster.
• Feb 19, 2016: Handle small payloads faster in level 1-3.
• Feb 19, 2016: Added faster level 2 + 3 compression modes.
• Feb 19, 2016:Rebalanced compression levels, so there is a more even progression in terms of compression. New default level is 5.
• Feb 14, 2016: Snappy: Merge upstream changes.
• Feb 14, 2016: Snappy: Fix aggressive skipping.
• Feb 14, 2016: Snappy: Update benchmark.
• Feb 13, 2016: Deflate: Fixed assembler problem that could lead to sub-optimal compression.
• Feb 12, 2016: Snappy: Added AMD64 SSE 4.2 optimizations to matching, which makes easy to compress material run faster. Typical speedup is around 25%.
• Feb 9, 2016: Added Snappy package fork. This version is 5-7% faster, much more on hard to compress content.
• Jan 30, 2016: Optimize level 1 to 3 by not considering static dictionary or storing uncompressed. ~4-5% speedup.
• Jan 16, 2016: Optimization on deflate level 1,2,3 compression.
• Jan 8 2016: MergeCL 18317: fix reading, writing of zip64 archives.
• Dec 8 2015: Make level 1 and -2 deterministic even if write size differs.
• Dec 8 2015: Split encoding functions, so hashing and matching can potentially be inlined. 1-3% faster on AMD64. 5% faster on other platforms.
• Dec 8 2015: Fixed rareone byte out-of bounds read. Please update!
• Nov 23 2015: Optimization on token writer. ~2-4% faster. Contributed by@dsnet.
• Nov 20 2015: Small optimization to bit writer on 64 bit systems.
• Nov 17 2015: Fixed out-of-bound errors if the underlying Writer returned an error. See#15.
• Nov 12 2015: Addedio.WriterTo support to gzip/inflate.
• Nov 11 2015: MergedCL 16669: archive/zip: enable overriding (de)compressors per file
• Oct 15 2015: Added skipping on uncompressible data. Random data speed up >5x.