Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork366
Zarr Visualization Roadmap (+ BOF CNG 2025)#3043
-
On May 2, 2025, at theCNG 2025, we held a birds of a feather session onZarr Visualization approaches. As an opening prompt, attendees were asked to share what their interests and experience is with Zarr visualization approaches. Attendees were familiar with or using @carbonplan/maps, have built in-house Zarr visualization platforms with pyramiding and caching, and were interested in using the GPU for visualization and 3 and 4-D mapping. In general, attendees agreed that visualizing Zarr should be "as-easy" as visualizing COGs, with interoperability for platforms such as QGIS, so Zarr datasets can be accessible to all levels of user. Summary of discussion
Roadmap for Zarr VisualizationI wrote up a roadmap that I felt represented the discussion about order of operations. I am open to feedback or modifications!
The spec should include:
There is no need for a server vs client-side debate as both are necessary for different use cases: Server-side can plug-in to existing tools via WMS and WMTS. Client-side enables power, serverless direct data interactions.
Open questions
|
BetaWas this translation helpful?Give feedback.
All reactions
👍 1❤️ 1
Replies: 2 comments 3 replies
-
Thanks for sharing this writeup Aimee!
Just to confirm, is the idea for this library to translate something likemorecantile Tiles or lat / lon coordinates to the appropriate chunks to index into a Zarr array? We'd still use zarr-python for the actual data reading? |
BetaWas this translation helpful?Give feedback.
All reactions
-
I suspect the IO layer of neuroglancer is the most performant web-based zarr IO implementation available.@jbms, do you have any estimate for how complex it would be to factor neuroglancers chunked IO into a separate package? |
BetaWas this translation helpful?Give feedback.
All reactions
-
I think this was a question about server-side --- for which there are already many implementations --- e.g. zarr-python, tensorstore, zarrs. Regarding multi-resolution --- that is certainly pretty critical. Even though it is oriented around microscopy data, you could use OME-zarr already to represent the multi-resolution levels, and perhaps just substitute suitable geo units in place of the ones currently supported by OME-zarr. For web client side application usage, Neuroglancer's I/O layer is not easily used separately. But also there is only a very small amount of code specific to zarr --- mostly it is just a matter of integrating zarr as a format into neuroglancer's I/O, caching, and multi-resolution volume interfaces. Adding support for geo transforms and other geo data types to Neuroglancer is something that I considered --- that would be an alternative approach to building a new viewer or adapting existing viewers to support 3-d or n-d data. Re icechunk: Currently neuroglancer supports the old msgpack-based icechunk format. I haven't yet updated it to support the new flatbuffers format and I imagine it may change significantly at least one more time in the near future, and once support for this or a later version is added I probably won't bother to keep supporting the old versions until it becomes stable. Depending on how urgently you want to actually use it, you might instead just wait to support it until it becomes more stable. |
BetaWas this translation helpful?Give feedback.
All reactions
👍 1
-
That was my understanding too. My hope is that any of those can achieve about equivalent performance for reading chunks. |
BetaWas this translation helpful?Give feedback.
All reactions
-
One thing that existing implementations don't support is efficientlytranscoding between different encodings of chunks, where there may be a more optimized path than fully decoding to raw uncompressed format and then re-encoding to the target format. For example, you might have a zarr array using some to-be-implemented jpeg codec for chunks, and want to transcode to some geo-oriented tile format that also supports jpeg encoding. With the existing libraries I'm aware of, you would have to fully decode the original jpeg chunks and then re-encode into the new jpeg tiles, which is both inefficient and lossy compared to just not decoding the jpeg at all, and merely copying the jpeg data into a new container format. |
BetaWas this translation helpful?Give feedback.
All reactions
🚀 2