Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

defining a zarr schema and validating meets that schema#2364

Unanswered
hardingnj asked this question inQ&A
Discussion options

Hi everyone,

We use zarr as a backend to store data generated by a pipeline. These data take the form of counts, and metadata.

We typically usexarray to write and load zarr stores intoxarray.Dataset objects.

As we begin to formalise, we realise we need a way to validate that a zarr store is what we are expecting. Specifically:

  • it contains the arrays we expect
  • the dtypes are as expected
  • the shapes are consistent, ie array B shares dimension 0 with dimension 1 of array A.
  • attributes are provided and meet expectations.
  • all blocks are present (no missing data)

I've looked at pydantic and AVRO, but they don't seem to do exactly what I want. Is anyone aware of any other solutions that would enable this? Or able to provide an example?

You must be logged in to vote

Replies: 2 comments 2 replies

Comment options

I solve this problem by making pydantic models for my zarr arrays / groups: seehttps://github.com/janelia-cellmap/pydantic-zarr. I'd be curious to hear your feedback on that library. it's definitely been extremely helpful for my own work.

You must be logged in to vote
1 reply
@hardingnj
Comment options

Thanks! Looks really helpful

Comment options

Since you are using Xarray, you may also want to take a look atxarray-schema.

You must be logged in to vote
1 reply
@hardingnj
Comment options

Great suggestion- wasn't aware of this...

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Category
Q&A
Labels
None yet
3 participants
@hardingnj@jhamman@d-v-b

[8]ページ先頭

©2009-2025 Movatter.jp