Commitd6d921e

fjansson

and

d-v-b

authored

fsspec path handling (#3343)

* fsspec path handlingProposed as fix for#3201. Some filesystems need the scheme as part of the path,while others don't. FsspecStore.from_url() throws an exception if the scheme isleft in the path, for any filesystem except http and https. However, the swift fsalso needs the scheme in the path. This commit removes the exception, rather thanadding more special cases.* fsspec path handling: remove test for scheme in path---------Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

1 parent61ca06b commitd6d921eCopy full SHA for d6d921e

File tree

2 files changed

-18

lines changed

src/zarr/storage
- _fsspec.py
tests/test_store
- test_fsspec.py

2 files changed

-18

lines changed

`‎src/zarr/storage/_fsspec.py‎`

Lines changed: 0 additions & 10 deletions

Original file line number	Diff line number	Diff line change
`@@ -138,10 +138,6 @@ def __init__(`
`138`	`138`	`category=ZarrUserWarning,`
`139`	`139`	`stacklevel=2,`
`140`	`140`	`)`
`141`		`-if"://"inpathandnotpath.startswith("http"):`
`142`		-# `not path.startswith("http")` is a special case for the http filesystem (¯\_(ツ)_/¯)
`143`		`-scheme,_=path.split("://",maxsplit=1)`
`144`		`-raiseValueError(f"path argument to FsspecStore must not include scheme ({scheme}://)")`
`145`	`141`
`146`	`142`	`@classmethod`
`147`	`143`	`deffrom_upath(`
`@@ -246,12 +242,6 @@ def from_url(`
`246`	`242`	`ifnotfs.async_impl:`
`247`	`243`	`fs=_make_async(fs)`
`248`	`244`
`249`		`-# fsspec is not consistent about removing the scheme from the path, so check and strip it here`
`250`		`-# https://github.com/fsspec/filesystem_spec/issues/1722`
`251`		`-if"://"inpathandnotpath.startswith("http"):`
`252`		-# `not path.startswith("http")` is a special case for the http filesystem (¯\_(ツ)_/¯)
`253`		`-path=fs._strip_protocol(path)`
`254`		`-`
`255`	`245`	`returncls(fs=fs,path=path,read_only=read_only,allowed_exceptions=allowed_exceptions)`
`256`	`246`
`257`	`247`	`defwith_read_only(self,read_only:bool=False)->FsspecStore:`

`‎tests/test_store/test_fsspec.py‎`

Lines changed: 0 additions & 8 deletions

Original file line number	Diff line number	Diff line change
`@@ -238,14 +238,6 @@ def test_from_upath(self) -> None:`
`238`	`238`	`assertresult.fs.asynchronous`
`239`	`239`	`assertresult.path==f"{test_bucket_name}/foo/bar"`
`240`	`240`
`241`		`-deftest_init_raises_if_path_has_scheme(self,store_kwargs:dict[str,Any])->None:`
`242`		`-# regression test for https://github.com/zarr-developers/zarr-python/issues/2342`
`243`		`-store_kwargs["path"]="s3://"+store_kwargs["path"]`
`244`		`-withpytest.raises(`
`245`		`-ValueError,match="path argument to FsspecStore must not include scheme .*"`
`246`		`- ):`
`247`		`-self.store_cls(**store_kwargs)`
`248`		`-`
`249`	`241`	`deftest_init_warns_if_fs_asynchronous_is_false(self)->None:`
`250`	`242`	`try:`
`251`	`243`	`fromfsspecimporturl_to_fs`

0 commit comments

Comments

(0)

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Commitd6d921e

File tree

2 files changed

2 files changed

`‎src/zarr/storage/_fsspec.py‎`

`‎tests/test_store/test_fsspec.py‎`

0 commit comments