Commit45756b0

K-Meech

and

d-v-b

authored

Document StoreLike values in user guide (#3480)

* document valid values for StoreLike in user guide* fix doctests for storelike examples* make links mkdocs style* fix user guide links for mkdocs* link all StoreLike refs to the user guide* add changes note* remove old storage rst file---------Co-authored-by: Davis Bennett <davis.v.bennett@gmail.com>

1 parent2d3d790 commit45756b0Copy full SHA for 45756b0

File tree

8 files changed

+159

-57

lines changed

changes
- 3303.doc.md
docs/user-guide
- storage.md
mkdocs.yml
src/zarr
- api
  - asynchronous.py
  - synchronous.py
- core
  - array.py
  - group.py
- storage
  - _common.py

8 files changed

+159

-57

lines changed

`‎changes/3303.doc.md‎`

Lines changed: 1 addition & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1 @@`
	`1`	`+Document different values of StoreLike with examples in the user guide.`

`‎docs/user-guide/storage.md‎`

Lines changed: 49 additions & 1 deletion

Original file line number	Diff line number	Diff line change
`@@ -12,7 +12,8 @@ Zarr-Python 3, stores must implement the abstract store API from`
`12`	`12`	`##Implicit Store Creation`
`13`	`13`
`14`	`14`	In most cases, it is not required to create a`Store` object explicitly. Passing a string
`15`		`-to Zarr's top level API will result in the store being created automatically:`
	`15`	`+(or other[StoreLike value](#storelike)) to Zarr's top level API will result in the store`
	`16`	`+being created automatically:`
`16`	`17`
`17`	`18`	```python exec="true" session="storage" source="above" result="ansi"
`18`	`19`	`import zarr`
`@@ -39,6 +40,53 @@ group = zarr.create_group(store=data)`
`39`	`40`	`print(group)`
`40`	`41`	```
`41`	`42`
	`43`	`+[](){#user-guide-store-like}`
	`44`	`+###StoreLike`
	`45`	`+`
	`46`	+`StoreLike` values can be:
	`47`	`+`
	`48`	+- a`Path` or string indicating a location on the local file system.
	`49`	`+ This will create a[local store](#local-store):`
	`50`	+```python exec="true" session="storage" source="above" result="ansi"
	`51`	`+ group= zarr.open_group(store='data/foo/bar')`
	`52`	`+print(group)`
	`53`	+```
	`54`	+```python exec="true" session="storage" source="above" result="ansi"
	`55`	`+from pathlibimport Path`
	`56`	`+ group= zarr.open_group(store=Path('data/foo/bar'))`
	`57`	`+print(group)`
	`58`	+```
	`59`	`+`
	`60`	`+- an FSSpec URI string, indicating a[remote store](#remote-store) location:`
	`61`	+```python exec="true" session="storage" source="above" result="ansi"
	`62`	`+ group= zarr.open_group(`
	`63`	`+store='s3://noaa-nwm-retro-v2-zarr-pds',`
	`64`	`+mode='r',`
	`65`	`+storage_options={'anon':True}`
	`66`	`+ )`
	`67`	`+print(group)`
	`68`	+```
	`69`	`+`
	`70`	`+- an empty dictionary or None, which will create a new[memory store](#memory-store):`
	`71`	+```python exec="true" session="storage" source="above" result="ansi"
	`72`	`+ group= zarr.create_group(store={})`
	`73`	`+print(group)`
	`74`	+```
	`75`	+```python exec="true" session="storage" source="above" result="ansi"
	`76`	`+ group= zarr.create_group(store=None)`
	`77`	`+print(group)`
	`78`	+```
	`79`	`+`
	`80`	+- a dictionary of string to[`Buffer`][zarr.abc.buffer.Buffer] mappings. This will
	`81`	`+ create a[memory store](#memory-store), using this dictionary as the`
	`82`	+[`store_dict` argument][zarr.storage.MemoryStore].
	`83`	`+`
	`84`	`+- an FSSpec[FSMap object](https://filesystem-spec.readthedocs.io/en/latest/api.html#fsspec.FSMap),`
	`85`	`+ which will create an[FsspecStore](#remote-store).`
	`86`	`+`
	`87`	+- a[`Store`][zarr.abc.store.Store] or[`StorePath`][zarr.storage.StorePath] -
	`88`	`+ see explicit store creation below.`
	`89`	`+`
`42`	`90`	`##Explicit Store Creation`
`43`	`91`
`44`	`92`	`In some cases, it may be helpful to create a store instance directly. Zarr-Python offers four`

`‎mkdocs.yml‎`

Lines changed: 1 addition & 0 deletions

Original file line number	Diff line number	Diff line change
`@@ -107,6 +107,7 @@ extra_css:`
`107`	`107`	`-overrides/stylesheets/extra.css`
`108`	`108`
`109`	`109`	`plugins:`
	`110`	`+ -autorefs`
`110`	`111`	`-search`
`111`	`112`	`-markdown-exec`
`112`	`113`	`-mkdocstrings:`

`‎src/zarr/api/asynchronous.py‎`

Lines changed: 36 additions & 22 deletions

Original file line number	Diff line number	Diff line change
`@@ -199,7 +199,9 @@ async def consolidate_metadata(`
`199`	`199`	`Parameters`
`200`	`200`	`----------`
`201`	`201`	`store : StoreLike`
`202`		`- The store-like object whose metadata you wish to consolidate.`
	`202`	`+ The store-like object whose metadata you wish to consolidate. See the`
	`203`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`204`	`+ for a description of all valid StoreLike values.`
`203`	`205`	`path : str, optional`
`204`	`206`	`A path to a group in the store to consolidate at. Only children`
`205`	`207`	`below that group will be consolidated.`
`@@ -293,7 +295,9 @@ async def load(`
`293`	`295`	`Parameters`
`294`	`296`	`----------`
`295`	`297`	`store : StoreLike`
`296`		`- Store or path to directory in file system or name of zip file.`
	`298`	`+ StoreLike object to open. See the`
	`299`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`300`	`+ for a description of all valid StoreLike values.`
`297`	`301`	`path : str or None, optional`
`298`	`302`	`The path within the store from which to load.`
`299`	`303`
`@@ -337,7 +341,9 @@ async def open(`
`337`	`341`	`Parameters`
`338`	`342`	`----------`
`339`	`343`	`store : StoreLike or None, default=None`
`340`		`- Store or path to directory in file system or name of zip file.`
	`344`	`+ StoreLike object to open. See the`
	`345`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`346`	`+ for a description of all valid StoreLike values.`
`341`	`347`	`mode : {'r', 'r+', 'a', 'w', 'w-'}, optional`
`342`	`348`	`Persistence mode: 'r' means read only (must exist); 'r+' means`
`343`	`349`	`read/write (must exist); 'a' means read/write (create if doesn't`
`@@ -421,7 +427,9 @@ async def save(`
`421`	`427`	`Parameters`
`422`	`428`	`----------`
`423`	`429`	`store : StoreLike`
`424`		`- Store or path to directory in file system or name of zip file.`
	`430`	`+ StoreLike object to open. See the`
	`431`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`432`	`+ for a description of all valid StoreLike values.`
`425`	`433`	`*args : ndarray`
`426`	`434`	`NumPy arrays with data to save.`
`427`	`435`	`zarr_format : {2, 3, None}, optional`
`@@ -457,7 +465,9 @@ async def save_array(`
`457`	`465`	`Parameters`
`458`	`466`	`----------`
`459`	`467`	`store : StoreLike`
`460`		`- Store or path to directory in file system or name of zip file.`
	`468`	`+ StoreLike object to open. See the`
	`469`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`470`	`+ for a description of all valid StoreLike values.`
`461`	`471`	`arr : ndarray`
`462`	`472`	`NumPy array with data to save.`
`463`	`473`	`zarr_format : {2, 3, None}, optional`
`@@ -513,7 +523,9 @@ async def save_group(`
`513`	`523`	`Parameters`
`514`	`524`	`----------`
`515`	`525`	`store : StoreLike`
`516`		`- Store or path to directory in file system or name of zip file.`
	`526`	`+ StoreLike object to open. See the`
	`527`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`528`	`+ for a description of all valid StoreLike values.`
`517`	`529`	`*args : ndarray`
`518`	`530`	`NumPy arrays with data to save.`
`519`	`531`	`zarr_format : {2, 3, None}, optional`
`@@ -662,7 +674,9 @@ async def group(`
`662`	`674`	`Parameters`
`663`	`675`	`----------`
`664`	`676`	`store : StoreLike or None, default=None`
`665`		`- Store or path to directory in file system or name of zip file.`
	`677`	`+ StoreLike object to open. See the`
	`678`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`679`	`+ for a description of all valid StoreLike values.`
`666`	`680`	`overwrite : bool, optional`
`667`	`681`	If True, delete any pre-existing data in `store` at `path` before
`668`	`682`	`creating the group.`
`@@ -724,7 +738,9 @@ async def create_group(`
`724`	`738`	`Parameters`
`725`	`739`	`----------`
`726`	`740`	`store : StoreLike`
`727`		`- Store or path to directory in file system or name of zip file.`
	`741`	`+ StoreLike object to open. See the`
	`742`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`743`	`+ for a description of all valid StoreLike values.`
`728`	`744`	`path : str, optional`
`729`	`745`	`Group path within store.`
`730`	`746`	`overwrite : bool, optional`
`@@ -780,17 +796,9 @@ async def open_group(`
`780`	`796`	`Parameters`
`781`	`797`	`----------`
`782`	`798`	`store : StoreLike or None, default=None`
`783`		`- Store or path to directory in file system or name of zip file.`
`784`		`-`
`785`		`- Strings are interpreted as paths on the local file system`
`786`		- and used as the ``root`` argument to [zarr.storage.LocalStore][].
`787`		`-`
`788`		- Dictionaries are used as the ``store_dict`` argument in
`789`		`- [zarr.storage.MemoryStore][].`
`790`		`-`
`791`		- By default (``store=None``) a new [zarr.storage.MemoryStore][]
`792`		`- is created.`
`793`		`-`
	`799`	`+ StoreLike object to open. See the`
	`800`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`801`	`+ for a description of all valid StoreLike values.`
`794`	`802`	`mode : {'r', 'r+', 'a', 'w', 'w-'}, optional`
`795`	`803`	`Persistence mode: 'r' means read only (must exist); 'r+' means`
`796`	`804`	`read/write (must exist); 'a' means read/write (create if doesn't`
`@@ -805,7 +813,9 @@ async def open_group(`
`805`	`813`	`path : str, optional`
`806`	`814`	`Group path within store.`
`807`	`815`	`chunk_store : StoreLike or None, default=None`
`808`		`- Store or path to directory in file system or name of zip file.`
	`816`	`+ Separate storage for chunks. See the`
	`817`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`818`	`+ for a description of all valid StoreLike values.`
`809`	`819`	`storage_options : dict`
`810`	`820`	`If using an fsspec URL to create the store, these will be passed to`
`811`	`821`	`the backend implementation. Ignored otherwise.`
`@@ -939,7 +949,9 @@ async def create(`
`939`	`949`	`Memory layout to be used within each chunk.`
`940`	`950`	If not specified, the ``array.order`` parameter in the global config will be used.
`941`	`951`	`store : StoreLike or None, default=None`
`942`		`- Store or path to directory in file system or name of zip file.`
	`952`	`+ StoreLike object to open. See the`
	`953`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`954`	`+ for a description of all valid StoreLike values.`
`943`	`955`	`synchronizer : object, optional`
`944`	`956`	`Array synchronizer.`
`945`	`957`	`overwrite : bool, optional`
`@@ -1250,7 +1262,9 @@ async def open_array(`
`1250`	`1262`	`Parameters`
`1251`	`1263`	`----------`
`1252`	`1264`	`store : StoreLike`
`1253`		`- Store or path to directory in file system or name of zip file.`
	`1265`	`+ StoreLike object to open. See the`
	`1266`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`1267`	`+ for a description of all valid StoreLike values.`
`1254`	`1268`	`zarr_version : {2, 3, None}, optional`
`1255`	`1269`	`The zarr format to use when saving. Deprecated in favor of zarr_format.`
`1256`	`1270`	`zarr_format : {2, 3, None}, optional`

`‎src/zarr/api/synchronous.py‎`

Lines changed: 42 additions & 24 deletions

Original file line number	Diff line number	Diff line change
`@@ -88,7 +88,9 @@ def consolidate_metadata(`
`88`	`88`	`Parameters`
`89`	`89`	`----------`
`90`	`90`	`store : StoreLike`
`91`		`- The store-like object whose metadata you wish to consolidate.`
	`91`	`+ The store-like object whose metadata you wish to consolidate. See the`
	`92`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`93`	`+ for a description of all valid StoreLike values.`
`92`	`94`	`path : str, optional`
`93`	`95`	`A path to a group in the store to consolidate at. Only children`
`94`	`96`	`below that group will be consolidated.`
`@@ -143,7 +145,9 @@ def load(`
`143`	`145`	`Parameters`
`144`	`146`	`----------`
`145`	`147`	`store : StoreLike`
`146`		`- Store or path to directory in file system or name of zip file.`
	`148`	`+ StoreLike object to open. See the`
	`149`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`150`	`+ for a description of all valid StoreLike values.`
`147`	`151`	`path : str or None, optional`
`148`	`152`	`The path within the store from which to load.`
`149`	`153`
`@@ -183,7 +187,9 @@ def open(`
`183`	`187`	`Parameters`
`184`	`188`	`----------`
`185`	`189`	`store : StoreLike or None, default=None`
`186`		`- Store or path to directory in file system or name of zip file.`
	`190`	`+ StoreLike object to open. See the`
	`191`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`192`	`+ for a description of all valid StoreLike values.`
`187`	`193`	`mode : {'r', 'r+', 'a', 'w', 'w-'}, optional`
`188`	`194`	`Persistence mode: 'r' means read only (must exist); 'r+' means`
`189`	`195`	`read/write (must exist); 'a' means read/write (create if doesn't`
`@@ -245,7 +251,9 @@ def save(`
`245`	`251`	`Parameters`
`246`	`252`	`----------`
`247`	`253`	`store : StoreLike`
`248`		`- Store or path to directory in file system or name of zip file.`
	`254`	`+ StoreLike object to open. See the`
	`255`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`256`	`+ for a description of all valid StoreLike values.`
`249`	`257`	`*args : ndarray`
`250`	`258`	`NumPy arrays with data to save.`
`251`	`259`	`zarr_format : {2, 3, None}, optional`
`@@ -279,7 +287,9 @@ def save_array(`
`279`	`287`	`Parameters`
`280`	`288`	`----------`
`281`	`289`	`store : StoreLike`
`282`		`- Store or path to directory in file system or name of zip file.`
	`290`	`+ StoreLike object to open. See the`
	`291`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`292`	`+ for a description of all valid StoreLike values.`
`283`	`293`	`arr : ndarray`
`284`	`294`	`NumPy array with data to save.`
`285`	`295`	`zarr_format : {2, 3, None}, optional`
`@@ -322,7 +332,9 @@ def save_group(`
`322`	`332`	`Parameters`
`323`	`333`	`----------`
`324`	`334`	`store : StoreLike`
`325`		`- Store or path to directory in file system or name of zip file.`
	`335`	`+ StoreLike object to open. See the`
	`336`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`337`	`+ for a description of all valid StoreLike values.`
`326`	`338`	`*args : ndarray`
`327`	`339`	`NumPy arrays with data to save.`
`328`	`340`	`zarr_format : {2, 3, None}, optional`
`@@ -413,7 +425,9 @@ def group(`
`413`	`425`	`Parameters`
`414`	`426`	`----------`
`415`	`427`	`store : StoreLike or None, default=None`
`416`		`- Store or path to directory in file system or name of zip file.`
	`428`	`+ StoreLike object to open. See the`
	`429`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`430`	`+ for a description of all valid StoreLike values.`
`417`	`431`	`overwrite : bool, optional`
`418`	`432`	If True, delete any pre-existing data in `store` at `path` before
`419`	`433`	`creating the group.`
`@@ -480,17 +494,9 @@ def open_group(`
`480`	`494`	`Parameters`
`481`	`495`	`----------`
`482`	`496`	`store : StoreLike or None, default=None`
`483`		`- Store or path to directory in file system or name of zip file.`
`484`		`-`
`485`		`- Strings are interpreted as paths on the local file system`
`486`		- and used as the ``root`` argument to [zarr.storage.LocalStore][].
`487`		`-`
`488`		- Dictionaries are used as the ``store_dict`` argument in
`489`		`- [zarr.storage.MemoryStore][].`
`490`		`-`
`491`		- By default (``store=None``) a new [zarr.storage.MemoryStore][]
`492`		`- is created.`
`493`		`-`
	`497`	`+ StoreLike object to open. See the`
	`498`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`499`	`+ for a description of all valid StoreLike values.`
`494`	`500`	`mode : {'r', 'r+', 'a', 'w', 'w-'}, optional`
`495`	`501`	`Persistence mode: 'r' means read only (must exist); 'r+' means`
`496`	`502`	`read/write (must exist); 'a' means read/write (create if doesn't`
`@@ -505,7 +511,9 @@ def open_group(`
`505`	`511`	`path : str, optional`
`506`	`512`	`Group path within store.`
`507`	`513`	`chunk_store : StoreLike or None, default=None`
`508`		`- Store or path to directory in file system or name of zip file.`
	`514`	`+ Separate storage for chunks. See the`
	`515`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`516`	`+ for a description of all valid StoreLike values.`
`509`	`517`	`storage_options : dict`
`510`	`518`	`If using an fsspec URL to create the store, these will be passed to`
`511`	`519`	`the backend implementation. Ignored otherwise.`
`@@ -570,7 +578,9 @@ def create_group(`
`570`	`578`	`Parameters`
`571`	`579`	`----------`
`572`	`580`	`store : StoreLike`
`573`		`- Store or path to directory in file system or name of zip file.`
	`581`	`+ StoreLike object to open. See the`
	`582`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`583`	`+ for a description of all valid StoreLike values.`
`574`	`584`	`path : str, optional`
`575`	`585`	`Group path within store.`
`576`	`586`	`overwrite : bool, optional`
`@@ -672,7 +682,9 @@ def create(`
`672`	`682`	`Memory layout to be used within each chunk.`
`673`	`683`	If not specified, the ``array.order`` parameter in the global config will be used.
`674`	`684`	`store : StoreLike or None, default=None`
`675`		`- Store or path to directory in file system or name of zip file.`
	`685`	`+ StoreLike object to open. See the`
	`686`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`687`	`+ for a description of all valid StoreLike values.`
`676`	`688`	`synchronizer : object, optional`
`677`	`689`	`Array synchronizer.`
`678`	`690`	`overwrite : bool, optional`
`@@ -832,7 +844,9 @@ def create_array(`
`832`	`844`	`Parameters`
`833`	`845`	`----------`
`834`	`846`	`store : StoreLike`
`835`		`- Store or path to directory in file system or name of zip file.`
	`847`	`+ StoreLike object to open. See the`
	`848`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`849`	`+ for a description of all valid StoreLike values.`
`836`	`850`	`name : str or None, optional`
`837`	`851`	The name of the array within the store. If ``name`` is ``None``, the array will be located
`838`	`852`	`at the root of the store.`
`@@ -996,7 +1010,9 @@ def from_array(`
`996`	`1010`	`Parameters`
`997`	`1011`	`----------`
`998`	`1012`	`store : StoreLike`
`999`		`- Store or path to directory in file system or name of zip file.`
	`1013`	`+ StoreLike object to open. See the`
	`1014`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`1015`	`+ for a description of all valid StoreLike values.`
`1000`	`1016`	`data : Array \| array-like`
`1001`	`1017`	`The array to copy.`
`1002`	`1018`	`write_data : bool, default True`
`@@ -1333,7 +1349,9 @@ def open_array(`
`1333`	`1349`	`Parameters`
`1334`	`1350`	`----------`
`1335`	`1351`	`store : StoreLike`
`1336`		`- Store or path to directory in file system or name of zip file.`
	`1352`	`+ StoreLike object to open. See the`
	`1353`	`+ [storage documentation in the user guide][user-guide-store-like]`
	`1354`	`+ for a description of all valid StoreLike values.`
`1337`	`1355`	`zarr_version : {2, 3, None}, optional`
`1338`	`1356`	`The zarr format to use when saving. Deprecated in favor of zarr_format.`
`1339`	`1357`	`zarr_format : {2, 3, None}, optional`

0 commit comments

Comments

(0)

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Commit45756b0

File tree

8 files changed

8 files changed

`‎changes/3303.doc.md‎`

`‎docs/user-guide/storage.md‎`

`‎mkdocs.yml‎`

`‎src/zarr/api/asynchronous.py‎`

`‎src/zarr/api/synchronous.py‎`

0 commit comments