Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: Implement ZEP 8 URL syntax support for zarr-python#3567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
ianhi wants to merge26 commits intozarr-developers:main
base:main
Choose a base branch
Loading
fromianhi:ian/jhamman/zep8

Conversation

@ianhi
Copy link
Contributor

Continuation of@jhamman's#3369 with addressing my review comments
Claude summary of the changes on top of Joe's PR:

  1. Format Propagation ✅
  • Zarr format segments (zarr2:, zarr3:) now propagate from URLs to array/group creation
  • Modified resolve_url_with_path() to return (Store, str, ZarrFormat | None)
  • Updated all API functions (create, open, open_array, open_group) to use URL format when user doesn't specify
  • 15 new tests for format propagation with comprehensive on-disk verification
  1. Nested Adapter Chains ✅
  • Implemented recursive URL resolution in ZipAdapter to support nested adapters
  • Added _create_nested_zip_store() method that handles nested ZEP 8 URLs
  • Enables ZEP 8 spec examples like file:outer.zip|zip:inner.zip|zip:data.zarr
  • 8 comprehensive tests covering arrays, groups, edge cases, and complex hierarchies
  1. Exception Handling Cleanup ✅
  • Removed all broad except Exception: handlers from _zep8.py
  • Simplified control flow by removing redundant try-except blocks
  • Only specific KeyError exceptions caught where appropriate
  • Parser exceptions now propagate naturally for easier debugging
  1. URL Logic Refactoring
  • Fixed s3+https scheme handling
  • Improved URL validation logic
  • Better separation of concerns between parsing and resolution
  1. Storage Options Validation
  • Enhanced storage_options handling and validation
  • Better error messages for invalid configurations

Test Coverage: 167 tests passing (1 skipped), up from ~150 tests


Missing ZEP 8 Features Analysis

Based on the ZEP 8 specification, here are the adapter schemes defined vs. implemented:

✅ Currently Implemented (9 adapters)

  • file: - FileSystemAdapter
  • memory: - MemoryAdapter
  • https:, http: - RemoteAdapter
  • s3:, s3+http:, s3+https: - S3Adapter (via RemoteAdapter)
  • gs: - GCSAdapter (via RemoteAdapter)
  • zip: - ZipAdapter
  • log: - LoggingAdapter (custom, not in spec)
  • zarr2:, zarr3:, zarr: - Format segments (handled by resolution layer)

❌ Missing from Spec (11 adapters)

Storage/Database Adapters:

  1. ocdbt: - OCDBT format (versioned KV store)
  2. icechunk: - Icechunk format (versioned Zarr store)

Compression Adapters:
3. gzip: - Transparent gzip decompression
4. zstd: - Transparent zstd decompression

Data Format Adapters:
5. n5: - N5 format support
6. tiff:, jpeg:, png:, bmp:, avif:, webp: - Image format adapters
7. neuroglancer-precomputed: - Neuroglancer format
8. json: - JSON pointer access

Utility Adapters:
9. byte-range:start-end - Byte range extraction
10. ..: - Parent directory traversal (for relative URLs)

Other Missing Features:

  • Relative URL pipeline syntax - Spec lines 489-543 (explicitly noted as not supported in zarr-python
    implementation notes)
  • Format auto-detection - Spec lines 420-443 (MAY support, optional feature)

joshmoore reacted with eyes emoji
jhammanand others added26 commitsAugust 11, 2025 13:13
- Add comprehensive ZEP 8 URL parsing and resolution system- Implement StoreAdapter ABC for extensible storage adapters- Add built-in adapters for file, memory, S3, GCS, HTTPS schemes- Support pipe-chained URLs like s3://bucket/data.zip|zip:|zarr3:- Add URLSegment parsing with validation- Integrate with zarr.open_group and zarr.open_array APIs- Include demo script and comprehensive test suite- Pass all existing tests + 35 new ZEP 8-specific tests
@github-actionsgithub-actionsbot added the needs release notesAutomatically applied to PRs which haven't added release notes labelNov 4, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

needs release notesAutomatically applied to PRs which haven't added release notes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ianhi@jhamman

[8]ページ先頭

©2009-2025 Movatter.jp