Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit7e21226

Browse files
authored
s2: Add LZ4 block converter (#748)
This allows converting compressed LZ4 blocks to S2 (or snappy) blocks without decompression.LZ4 -> S2 seems to be same size on average.LZ4 -> Snappy is usually worse.## Single threaded performanceSpeed (excluding LZ4 encoding):```BenchmarkLZ4Converter_ConvertBlock/html-32 28237 42827 ns/op2390.99 MB/s 559.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/urls-32 2138 541816 ns/op1295.80 MB/s -3943 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/jpg-32 514826 2328 ns/op52874.24 MB/s 482.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/jpg_200b-32 34821668 33.48 ns/op5973.00 MB/s 2.000 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/pdf-32 198241 5975 ns/op17136.81 MB/s 136.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/html4-32 7002 173440 ns/op2361.63 MB/s 1840 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1-32 5940 196951 ns/op 772.22 MB/s 106.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt2-32 6656 177228 ns/op 706.32 MB/s -1427 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt3-32 2355 510435 ns/op 836.06 MB/s 384.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt4-32 1700 694444 ns/op 693.88 MB/s -9125 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/pb-32 37118 32141 ns/op3689.60 MB/s 1.000 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/gaviota-32 6961 172253 ns/op1070.05 MB/s 9303 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1_128b-32 19923691 59.82 ns/op2139.66 MB/s 0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1_1000b-32 3180837 375.2 ns/op2665.40 MB/s 16.00 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1_10000b-32 184214 6350 ns/op1574.70 MB/s 90.00 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1_20000b-32 74031 15521 ns/op1288.54 MB/s -5.000 b_saved 0 B/op 0 allocs/op```Assembly speed (amd64)```BenchmarkLZ4Converter_ConvertBlock/html-32 47457 24463 ns/op4185.89 MB/s 559.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/urls-32 3506 330277 ns/op2125.75 MB/s -3943 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/jpg-32 450177 2718 ns/op45294.89 MB/s 482.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/jpg_200b-32 76887589 15.52 ns/op12887.16 MB/s 2.000 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/pdf-32 279540 4322 ns/op23694.21 MB/s 136.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/html4-32 10000 107485 ns/op3810.75 MB/s 1840 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1-32 10000 117764 ns/op1291.47 MB/s 106.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt2-32 10000 100578 ns/op1244.60 MB/s -1427 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt3-32 3793 313021 ns/op1363.34 MB/s 384.0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt4-32 2988 399888 ns/op1204.99 MB/s -9125 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/pb-32 57486 19277 ns/op6151.76 MB/s 1.000 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/gaviota-32 10000 115641 ns/op1593.90 MB/s 9303 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1_128b-32 38400122 31.21 ns/op4101.10 MB/s 0 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1_1000b-32 6509028 179.9 ns/op5559.38 MB/s 16.00 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1_10000b-32 368212 3244 ns/op3082.28 MB/s 90.00 b_saved 0 B/op 0 allocs/opBenchmarkLZ4Converter_ConvertBlock/txt1_20000b-32 141013 8303 ns/op2408.69 MB/s -5.000 b_saved 0 B/op 0 allocs/op```Reference compression speed:```BenchmarkCompressBlockReference/html/default-32 14070 82449 ns/op 1241.98 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/urls/default-32 1215 970890 ns/op 723.14 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/jpg/default-32 193770 5904 ns/op 20849.30 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/jpg_200b/default-32 8297767 144.1 ns/op 1387.77 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/pdf/default-32 94203 12694 ns/op 8066.76 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/html4/default-32 12174 97969 ns/op 4180.90 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1/default-32 3613 333851 ns/op 455.56 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt2/default-32 4683 260579 ns/op 480.39 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt3/default-32 1268 947209 ns/op 450.54 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt4/default-32 1083 1097426 ns/op 439.08 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/pb/default-32 18357 64771 ns/op 1830.90 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/gaviota/default-32 3942 295275 ns/op 624.23 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1_128b/default-32 11448295 105.7 ns/op 1210.45 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1_1000b/default-32 1000000 1021 ns/op 979.26 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1_10000b/default-32 116739 10114 ns/op 988.68 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1_20000b/default-32 49216 23409 ns/op 854.39 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/html/better-32 6649 174667 ns/op 586.26 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/urls/better-32 627 1905706 ns/op 368.41 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/jpg/better-32 52425 22783 ns/op 5402.88 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/jpg_200b/better-32 2772865 433.3 ns/op 461.61 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/pdf/better-32 9210 127051 ns/op 805.97 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/html4/better-32 5835 201146 ns/op 2036.33 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1/better-32 2034 566702 ns/op 268.38 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt2/better-32 2386 500580 ns/op 250.07 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt3/better-32 758 1556541 ns/op 274.17 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt4/better-32 591 2013515 ns/op 239.31 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/pb/better-32 7836 155117 ns/op 764.51 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/gaviota/better-32 2473 484975 ns/op 380.06 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1_128b/better-32 4322678 275.5 ns/op 464.59 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1_1000b/better-32 468687 2533 ns/op 394.76 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1_10000b/better-32 49606 23720 ns/op 421.59 MB/s 0 B/op 0 allocs/opBenchmarkCompressBlockReference/txt1_20000b/better-32 14823 81300 ns/op 246.00 MB/s 0 B/op 0 allocs/op```Size comparisons (using Go lz4 encoder):```=== RUN TestLZ4Converter_ConvertBlock/html lz4convert_test.go:42: input size: 102400 lz4convert_test.go:43: lz4 size: 21195 lz4convert_test.go:60: lz4->snappy size: 21828 lz4convert_test.go:79: lz4->s2 size: 20636 lz4convert_test.go:91: s2 (default) size: 20865 lz4convert_test.go:95: s2 (better) size: 18969 lz4convert_test.go:97: lz4 -> s2 bytes saved: 559 lz4convert_test.go:98: lz4 -> snappy bytes saved: -633 lz4convert_test.go:99: data -> s2 (default) bytes saved: 330 lz4convert_test.go:100: data -> s2 (better) bytes saved: 2226 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -229 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 1667 --- PASS: TestLZ4Converter_ConvertBlock/html (0.00s)=== RUN TestLZ4Converter_ConvertBlock/urls lz4convert_test.go:42: input size: 702087 lz4convert_test.go:43: lz4 size: 292514 lz4convert_test.go:60: lz4->snappy size: 297926 lz4convert_test.go:79: lz4->s2 size: 296457 lz4convert_test.go:91: s2 (default) size: 286538 lz4convert_test.go:95: s2 (better) size: 248076 lz4convert_test.go:97: lz4 -> s2 bytes saved: -3943 lz4convert_test.go:98: lz4 -> snappy bytes saved: -5412 lz4convert_test.go:99: data -> s2 (default) bytes saved: 5976 lz4convert_test.go:100: data -> s2 (better) bytes saved: 44438 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 9919 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 48381 --- PASS: TestLZ4Converter_ConvertBlock/urls (0.01s)=== RUN TestLZ4Converter_ConvertBlock/jpg lz4convert_test.go:42: input size: 123093 lz4convert_test.go:43: lz4 size: 123522 lz4convert_test.go:60: lz4->snappy size: 123040 lz4convert_test.go:79: lz4->s2 size: 123040 lz4convert_test.go:91: s2 (default) size: 123097 lz4convert_test.go:95: s2 (better) size: 123097 lz4convert_test.go:97: lz4 -> s2 bytes saved: 482 lz4convert_test.go:98: lz4 -> snappy bytes saved: 482 lz4convert_test.go:99: data -> s2 (default) bytes saved: 425 lz4convert_test.go:100: data -> s2 (better) bytes saved: 425 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -57 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: -57 --- PASS: TestLZ4Converter_ConvertBlock/jpg (0.00s)=== RUN TestLZ4Converter_ConvertBlock/jpg_200b lz4convert_test.go:42: input size: 200 lz4convert_test.go:43: lz4 size: 155 lz4convert_test.go:60: lz4->snappy size: 153 lz4convert_test.go:79: lz4->s2 size: 153 lz4convert_test.go:91: s2 (default) size: 153 lz4convert_test.go:95: s2 (better) size: 147 lz4convert_test.go:97: lz4 -> s2 bytes saved: 2 lz4convert_test.go:98: lz4 -> snappy bytes saved: 2 lz4convert_test.go:99: data -> s2 (default) bytes saved: 2 lz4convert_test.go:100: data -> s2 (better) bytes saved: 8 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 0 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 6 --- PASS: TestLZ4Converter_ConvertBlock/jpg_200b (0.00s)=== RUN TestLZ4Converter_ConvertBlock/pdf lz4convert_test.go:42: input size: 102400 lz4convert_test.go:43: lz4 size: 83152 lz4convert_test.go:60: lz4->snappy size: 83428 lz4convert_test.go:79: lz4->s2 size: 83016 lz4convert_test.go:91: s2 (default) size: 84199 lz4convert_test.go:95: s2 (better) size: 82884 lz4convert_test.go:97: lz4 -> s2 bytes saved: 136 lz4convert_test.go:98: lz4 -> snappy bytes saved: -276 lz4convert_test.go:99: data -> s2 (default) bytes saved: -1047 lz4convert_test.go:100: data -> s2 (better) bytes saved: 268 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -1183 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 132 --- PASS: TestLZ4Converter_ConvertBlock/pdf (0.00s)=== RUN TestLZ4Converter_ConvertBlock/html4 lz4convert_test.go:42: input size: 409600 lz4convert_test.go:43: lz4 size: 81908 lz4convert_test.go:60: lz4->snappy size: 84886 lz4convert_test.go:79: lz4->s2 size: 80068 lz4convert_test.go:91: s2 (default) size: 20867 lz4convert_test.go:95: s2 (better) size: 18979 lz4convert_test.go:97: lz4 -> s2 bytes saved: 1840 lz4convert_test.go:98: lz4 -> snappy bytes saved: -2978 lz4convert_test.go:99: data -> s2 (default) bytes saved: 61041 lz4convert_test.go:100: data -> s2 (better) bytes saved: 62929 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 59201 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 61089 --- PASS: TestLZ4Converter_ConvertBlock/html4 (0.00s)=== RUN TestLZ4Converter_ConvertBlock/txt1 lz4convert_test.go:42: input size: 152089 lz4convert_test.go:43: lz4 size: 79672 lz4convert_test.go:60: lz4->snappy size: 79567 lz4convert_test.go:79: lz4->s2 size: 79566 lz4convert_test.go:91: s2 (default) size: 85931 lz4convert_test.go:95: s2 (better) size: 71608 lz4convert_test.go:97: lz4 -> s2 bytes saved: 106 lz4convert_test.go:98: lz4 -> snappy bytes saved: 105 lz4convert_test.go:99: data -> s2 (default) bytes saved: -6259 lz4convert_test.go:100: data -> s2 (better) bytes saved: 8064 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -6365 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 7958 --- PASS: TestLZ4Converter_ConvertBlock/txt1 (0.00s)=== RUN TestLZ4Converter_ConvertBlock/txt2 lz4convert_test.go:42: input size: 125179 lz4convert_test.go:43: lz4 size: 70801 lz4convert_test.go:60: lz4->snappy size: 72231 lz4convert_test.go:79: lz4->s2 size: 72228 lz4convert_test.go:91: s2 (default) size: 79572 lz4convert_test.go:95: s2 (better) size: 65938 lz4convert_test.go:97: lz4 -> s2 bytes saved: -1427 lz4convert_test.go:98: lz4 -> snappy bytes saved: -1430 lz4convert_test.go:99: data -> s2 (default) bytes saved: -8771 lz4convert_test.go:100: data -> s2 (better) bytes saved: 4863 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -7344 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 6290 --- PASS: TestLZ4Converter_ConvertBlock/txt2 (0.00s)=== RUN TestLZ4Converter_ConvertBlock/txt3 lz4convert_test.go:42: input size: 426754 lz4convert_test.go:43: lz4 size: 207038 lz4convert_test.go:60: lz4->snappy size: 206693 lz4convert_test.go:79: lz4->s2 size: 206654 lz4convert_test.go:91: s2 (default) size: 220380 lz4convert_test.go:95: s2 (better) size: 184936 lz4convert_test.go:97: lz4 -> s2 bytes saved: 384 lz4convert_test.go:98: lz4 -> snappy bytes saved: 345 lz4convert_test.go:99: data -> s2 (default) bytes saved: -13342 lz4convert_test.go:100: data -> s2 (better) bytes saved: 22102 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -13726 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 21718 --- PASS: TestLZ4Converter_ConvertBlock/txt3 (0.01s)=== RUN TestLZ4Converter_ConvertBlock/txt4 lz4convert_test.go:42: input size: 481861 lz4convert_test.go:43: lz4 size: 277731 lz4convert_test.go:60: lz4->snappy size: 286863 lz4convert_test.go:79: lz4->s2 size: 286856 lz4convert_test.go:91: s2 (default) size: 318193 lz4convert_test.go:95: s2 (better) size: 264987 lz4convert_test.go:97: lz4 -> s2 bytes saved: -9125 lz4convert_test.go:98: lz4 -> snappy bytes saved: -9132 lz4convert_test.go:99: data -> s2 (default) bytes saved: -40462 lz4convert_test.go:100: data -> s2 (better) bytes saved: 12744 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -31337 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 21869 --- PASS: TestLZ4Converter_ConvertBlock/txt4 (0.01s)=== RUN TestLZ4Converter_ConvertBlock/pb lz4convert_test.go:42: input size: 118588 lz4convert_test.go:43: lz4 size: 19003 lz4convert_test.go:60: lz4->snappy size: 21130 lz4convert_test.go:79: lz4->s2 size: 19002 lz4convert_test.go:91: s2 (default) size: 18603 lz4convert_test.go:95: s2 (better) size: 17686 lz4convert_test.go:97: lz4 -> s2 bytes saved: 1 lz4convert_test.go:98: lz4 -> snappy bytes saved: -2127 lz4convert_test.go:99: data -> s2 (default) bytes saved: 400 lz4convert_test.go:100: data -> s2 (better) bytes saved: 1317 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 399 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 1316 --- PASS: TestLZ4Converter_ConvertBlock/pb (0.00s)=== RUN TestLZ4Converter_ConvertBlock/gaviota lz4convert_test.go:42: input size: 184320 lz4convert_test.go:43: lz4 size: 71749 lz4convert_test.go:60: lz4->snappy size: 63392 lz4convert_test.go:79: lz4->s2 size: 62446 lz4convert_test.go:91: s2 (default) size: 65016 lz4convert_test.go:95: s2 (better) size: 55395 lz4convert_test.go:97: lz4 -> s2 bytes saved: 9303 lz4convert_test.go:98: lz4 -> snappy bytes saved: 8357 lz4convert_test.go:99: data -> s2 (default) bytes saved: 6733 lz4convert_test.go:100: data -> s2 (better) bytes saved: 16354 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -2570 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 7051 --- PASS: TestLZ4Converter_ConvertBlock/gaviota (0.00s)=== RUN TestLZ4Converter_ConvertBlock/txt1_128b lz4convert_test.go:42: input size: 128 lz4convert_test.go:43: lz4 size: 84 lz4convert_test.go:60: lz4->snappy size: 84 lz4convert_test.go:79: lz4->s2 size: 84 lz4convert_test.go:91: s2 (default) size: 80 lz4convert_test.go:95: s2 (better) size: 76 lz4convert_test.go:97: lz4 -> s2 bytes saved: 0 lz4convert_test.go:98: lz4 -> snappy bytes saved: 0 lz4convert_test.go:99: data -> s2 (default) bytes saved: 4 lz4convert_test.go:100: data -> s2 (better) bytes saved: 8 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 4 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 8 --- PASS: TestLZ4Converter_ConvertBlock/txt1_128b (0.00s)=== RUN TestLZ4Converter_ConvertBlock/txt1_1000b lz4convert_test.go:42: input size: 1000 lz4convert_test.go:43: lz4 size: 807 lz4convert_test.go:60: lz4->snappy size: 791 lz4convert_test.go:79: lz4->s2 size: 791 lz4convert_test.go:91: s2 (default) size: 772 lz4convert_test.go:95: s2 (better) size: 744 lz4convert_test.go:97: lz4 -> s2 bytes saved: 16 lz4convert_test.go:98: lz4 -> snappy bytes saved: 16 lz4convert_test.go:99: data -> s2 (default) bytes saved: 35 lz4convert_test.go:100: data -> s2 (better) bytes saved: 63 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: 19 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 47 --- PASS: TestLZ4Converter_ConvertBlock/txt1_1000b (0.00s)=== RUN TestLZ4Converter_ConvertBlock/txt1_10000b lz4convert_test.go:42: input size: 10000 lz4convert_test.go:43: lz4 size: 6969 lz4convert_test.go:60: lz4->snappy size: 6879 lz4convert_test.go:79: lz4->s2 size: 6879 lz4convert_test.go:91: s2 (default) size: 6931 lz4convert_test.go:95: s2 (better) size: 6216 lz4convert_test.go:97: lz4 -> s2 bytes saved: 90 lz4convert_test.go:98: lz4 -> snappy bytes saved: 90 lz4convert_test.go:99: data -> s2 (default) bytes saved: 38 lz4convert_test.go:100: data -> s2 (better) bytes saved: 753 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -52 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 663 --- PASS: TestLZ4Converter_ConvertBlock/txt1_10000b (0.00s)=== RUN TestLZ4Converter_ConvertBlock/txt1_20000b lz4convert_test.go:42: input size: 20000 lz4convert_test.go:43: lz4 size: 12750 lz4convert_test.go:60: lz4->snappy size: 12755 lz4convert_test.go:79: lz4->s2 size: 12755 lz4convert_test.go:91: s2 (default) size: 13513 lz4convert_test.go:95: s2 (better) size: 11489 lz4convert_test.go:97: lz4 -> s2 bytes saved: -5 lz4convert_test.go:98: lz4 -> snappy bytes saved: -5 lz4convert_test.go:99: data -> s2 (default) bytes saved: -763 lz4convert_test.go:100: data -> s2 (better) bytes saved: 1261 lz4convert_test.go:101: direct data -> s2 (default) compared to converted from lz4: -758 lz4convert_test.go:102: direct data -> s2 (better) compared to converted from lz4: 1266```
1 parent0793ca1 commit7e21226

File tree

14 files changed

+9575
-7112
lines changed

14 files changed

+9575
-7112
lines changed

‎huff0/decompress.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,7 @@ func ReadTable(in []byte, s *Scratch) (s2 *Scratch, remain []byte, err error) {
6161
b,err:=fse.Decompress(in[:iSize],s.fse)
6262
s.fse.Out=nil
6363
iferr!=nil {
64-
returns,nil,err
64+
returns,nil,fmt.Errorf("fse decompress returned: %w",err)
6565
}
6666
iflen(b)>255 {
6767
returns,nil,errors.New("corrupt input: output table too large")

‎internal/lz4ref/LICENSE‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
Copyright (c) 2015, Pierre Curto
2+
All rights reserved.
3+
4+
Redistribution and use in source and binary forms, with or without
5+
modification, are permitted provided that the following conditions are met:
6+
7+
* Redistributions of source code must retain the above copyright notice, this
8+
list of conditions and the following disclaimer.
9+
10+
* Redistributions in binary form must reproduce the above copyright notice,
11+
this list of conditions and the following disclaimer in the documentation
12+
and/or other materials provided with the distribution.
13+
14+
* Neither the name of xxHash nor the names of its
15+
contributors may be used to endorse or promote products derived from
16+
this software without specific prior written permission.
17+
18+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
19+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
20+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
21+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
22+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
23+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
24+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
25+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
26+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
27+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
28+

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp