Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork366
Use unsigned bytes to back Buffer#2738
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
PS, going to ignore tests for now. |
This makes compressors consistent with v2, and buffers consistents with`bytes` types.Fixeszarr-developers#2735
I just fixed uploading code coverage for the GPU tests at#2767, so hopefully when I merge in |
maxrjones commentedApr 11, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Hi@QuLogic and@dstansby, thanks for working on this! Could I help at all with resolving the codecov issue? I'd be super excited to see this completed to enable further trying out imagecodecs with Zarr V3 (e.g.,https://github.com/maxrjones/virtual-tiff/pull/4). |
(clicking update branch in case that fixes it) |
Closing and reopening to hopefully fix RTD (it has a |
Ok better RTD checked out the code 😌 |
Looks like Codecov got stuck somewhere. Let's try another close/reopen |
I don't think that the codecov/patch target will be hit unless the overall coverage actually increases, rather than remains constant. The issue is that two of the lines changed (L87 and L110) are not tested in main either -https://app.codecov.io/gh/zarr-developers/zarr-python/blob/main/src%2Fzarr%2Fcore%2Fbuffer%2Fgpu.py, so the PR isn't decreasing code coverage but still will not hit the 80% target for code coverage for patches (i.e., all new/modified lines of code). Do you prefer to ignore the codecov/patch target or to include unit tests for adding and creating zero length buffers? |
If someone can add some tests, that would be welcome What was concerning to me was one Codecov job says "failing after 1s". Though this may be a non-issue and just the report percentage (as you suggested) |
I added test coverage for the GPU buffer in#2978. However, the test for the GPU buffer with the sharding codec is failing. Debugging that failure would require a cuda-compatable GPU but I don't have one easily accessible. I am wondering if you'd consider merging this PR prior to addressing the failures in#2978 for a few reasons:
Thanks for your consideration, and@d-v-b for the helpful discussion and recommendation to post this suggestion at the Zarr-Python dev call today! |
thanks for pushing this along@maxrjones. Based on the changes in this PR, I'm happy merging this regardless of what codecov says at the moment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
to repeat what I said in thelinked issue, the particular numpy dtype we use in our buffer implementation should be considered an implementation detail by consumers. But for the same reason we can alter that dtype as needed, e.g. to make life easier for imagecodecs users, which is what this PR does. I'm going to approve and merge this despite the low test coverage because adding those tests depends on getting the gpu buffer to work with sharding, which is IMO out of scope for this PR.
bb55f0c intozarr-developers:mainUh oh!
There was an error while loading.Please reload this page.
This makes compressors consistent with v2, and buffers consistents with`bytes` types.Fixeszarr-developers#2735Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>Co-authored-by: David Stansby <dstansby@gmail.com>Co-authored-by: jakirkham <jakirkham@gmail.com>
This adds compatibility for zarr-python 3.0.7, which changed the dtype for representing bytes inzarr-developers/zarr-python#2738.Authors: - Tom Augspurger (https://github.com/TomAugspurger)Approvers: - Tianyu Liu (https://github.com/kingcrimsontianyu) - Benjamin Zaitlen (https://github.com/quasiben)URL:#699
* Avoid memory copy in obstore write* Add as_bytes_like method to Buffer* Add changelog entry* No need to take unsigned bytes view following#2738* Change method name to `as_buffer_like`---------Co-authored-by: jakirkham <jakirkham@gmail.com>Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>
This makes compressors consistent with v2, and seems more correct than signed bytes.
Fixes#2735
Note that this does not implement accepting both signed and unsigned, as there wasn't a conclusion in#2735 about that.
TODO:
docs/user-guide/*.rstdocs/release-notes.rst