- Notifications
You must be signed in to change notification settings - Fork153
Allow full unfiltering for partial data (enum-based approach)#664
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:master
Are you sure you want to change the base?
Conversation
The refactoring in this commit is desirable to support shifting by adifferent `discard_size` (i.e. making that independent from`self.prev_start`).This commit also opportunistically adds an `assert!` that ensures thatshifting left won't accidentally clobber the immutable "lookback"window.
The refactoring in this commit is desirable because:* Main reason: To support calling the extracted function from another place in a follow-up commit.* Secondary reason: To improve readability a little bit, by making `fn unfilter_curr_row` slightly less noisy / more focused on its core functionality, which is: - extracting `prev_row`, and `row` - calling `unfilter` - updating `current_start` and `prev_start`
This commit makes the following two changes:* Falls back to unfiltering the current row out-of-place if an `UnexpectedEof` error is encountered when in-place bytes cannot yet be mutated. Thisfixesimage-rs#639. See the following changes: - `test_partial_decode_success` in `tests/partial_decode.rs` - Changes in `fn next_raw_interlaced_row` method of `Reader` in `src/decoder/mod.rs` - Changes in `src/decoder/unfiltering_buffer.rs`* Ensures that `finish_compressed_chunks` is always called (even if all rows have already been successfully decoded). This is required to deterministically report errors, regardless of how the input is buffered. The other changes above would have resulted in fuzzing failures without fixing this. See the following changes: - `test_partial_decode_with_unterminated_redundant_zlib_tail` in `tests/partial_decode.rs` - Changes in `fn finish_decoding` method of `Reader` in `src/decoder/mod.rs` - Changes of `src/decoder/read_decoder.rs`
anforowicz commentedDec 10, 2025
There are someCIFuzz failures, but they seem to be caused by some infrastructure problems ( |
197g commentedDec 10, 2025
Did you use the corpus from oss-fuzz? For what it's worth the fuzz failure on the other PR is also somewhat spurious, it's a highly specific condition and fuzzing locally never found the same fault for me either (even after 64-threads for a day, it's just peanuts). |
anforowicz commentedDec 10, 2025
I've just run Would you have instructions for using oss-fuzz? (We may want to stash them into
Without changing |
197g commentedDec 10, 2025
Sorry, I've always relied on the API job for this and hoped you'd be closer to the source 😄 . For some concrete failures there are the failures on the mailing list but I would like to know how to retrieve the full corpus myself. |
197g commentedDec 11, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Update: the fuzzer results are now running and look on first site like a genuine detection. The first assert I see for that method is assert!(output_position <= output.len()); |
Uh oh!
There was an error while loading.Please reload this page.
Hello!
This PRfixes#639. This is an alternative approach to@197g's#640 - the two main differences are:
prev_start: usizeis replaced withprev_row: PrevRowwherePrevRowis anenumwith separateNone,InPlace, andScratchvariants. I think this results in easier to read code then imposing a special meaning on certain values of offsets (e.g.self.prev_start == self.curr_start=> no previous row; or havingself.prev_startsometimes trailself.curr_startbyrowlen-1and sometimes point to mutable scratch space in the same buffer ascurr_start).PTAL?