Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork34.2k
gh-145607: Ensure BIG_DATA has two compressed blocks in test_bz2#145730
gh-145607: Ensure BIG_DATA has two compressed blocks in test_bz2#145730hugovk merged 1 commit intopython:mainfrom
Conversation
emmatyping commentedMar 10, 2026
Oh also I ran the test in a loop with random file orderings for a number of minutes to ensure that this actually fixes the bug. |
emmatyping commentedMar 10, 2026
!buildbot s390x RHEL9 |
bedevere-bot commentedMar 10, 2026
🤖 New build scheduled with the buildbot fleet by@emmatyping for commitf9e9d59 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F145730%2Fmerge The command will test the builders whose names match following regular expression: The builders matched are:
|
19676e5 intopython:mainUh oh!
There was an error while loading.Please reload this page.
Thanks@emmatyping for the PR, and@hugovk for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14. |
hugovk commentedMar 10, 2026
Thank you! |
pythonGH-145730)(cherry picked from commit 19676e5fc28bdee8325a062a31ddeee60960cf75)Co-authored-by: Emma Smith <emma@emmatyping.dev>
pythonGH-145730)(cherry picked from commit 19676e5fc28bdee8325a062a31ddeee60960cf75)Co-authored-by: Emma Smith <emma@emmatyping.dev>
GH-145733 is a backport of this pull request to the3.14 branch. |
GH-145734 is a backport of this pull request to the3.13 branch. |
Uh oh!
There was an error while loading.Please reload this page.
The
BIG_DATAvariable intest_bz2will sometimes only have one block of compressed data instead of the two it is intended to contain due to the value depending on the order of listing files. To fix this, we stream in input data and check for multiple block header magic strings to ensure that there are at least 2 block headers.This is an unfortunate hack, but the bz2 module doesn't expose flushing in calls to
BZ2Compressor.compress()so we cannot directly generate two blocks. I'm also not certain if block headers can span the output to.compress(), so conservatively I have the code scan the entire set of data, which should only need to happen a few times at test setup time. There's probably a few ways to optimize this code, but I wanted to keep it simple to get it in before the alpha tomorrow. I specifically did not want to use BZ2File as I wanted the module setup code to be as minimal as possible in case the implementation is broken.