Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork366
Compatibility policy for zarr-python v3#2097
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
-
Apologies if this has been discussed already, but is there policy written down anywhere with respect tocode compatibility written against zarr 2.x? Concretely, I wonder if making this snippet using xarray is a design goal of zarr-python v3? importxarrayasxrimportpathlibimportshutilp=pathlib.Path("out.zarr")shutil.rmtree(p,ignore_errors=True)ds=xr.tutorial.open_dataset("air_temperature")ds.to_zarr(p) That runs fine with zarr 2.18.2; Should it work with zarr-python 3.x without modification to xarray? To give you an idea of the kinds of changes needed to make this work:
There's some additional examples of this in some issues (#2023,#2021) One point worth discussing: is there value in code-level compatibility if zarr-python 3.x is (presumably) writing zarr-v3 by default? IMO yes, but perhaps not. |
BetaWas this translation helpful?Give feedback.
All reactions
Replies: 1 comment 2 replies
-
Short answer, if we wanted to preserve code-level compatibility, we would be targeting a 2.x release and not a 3.x release. The longer answer is that supporting zarr version 2and zarr version 3and preserving code compatibility with zarr-python v2 (which closely models zarr version 2) is not an easy path. So far, zarr-python 3.x has optimized for supporting zarr version 2 and zarr version 3, at the cost of changing zarr-python 2.x APIs. I think these are the right priorities (but others might disagree). Keep in mind that zarr v3 is a different format from zarr v2. Expecting old APIs to work for a new format seems like a stretch. That doesn't mean we should wantonly break stuff, and we are open to a variety of solutions for minimizing the changes. The simplest thing is for users who depend on zarr-python 2.x APIs to pin their dependencies to that version of zarr-python. More complicated solutions could be including all of zarr-python 2.x inside a v2 namespace in zarr-python 3.x, or publishing zarr-python 2.x as a stand-alone |
BetaWas this translation helpful?Give feedback.
All reactions
-
It's an interesting thought exercise for what has a longer tail: software or data files? :) Overall I think I agree: Prioritize APIs that make sense for Zarr v3, but don't wantonly break stuff. To make this a bit more concrete I published#2098 as a draft of what it takes to get |
BetaWas this translation helpful?Give feedback.
All reactions
👍 1
-
I have created#2102 to help address some of these questions. Contributions / feedback on that PR is welcome. |
BetaWas this translation helpful?Give feedback.