Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Add chunks='auto' support for cftime datasets#10527

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Draft
charles-turner-1 wants to merge9 commits intopydata:main
base:main
Choose a base branch
Loading
fromcharles-turner-1:autochunk-cftime

Conversation

charles-turner-1
Copy link

  • Closes #xxxx
  • Tests added

dcherian reacted with heart emoji
@welcomeWelcome
Copy link

welcomebot commentedJul 13, 2025

Thank you for opening this pull request! It may take us a few days to respond here, so thank you for being patient.
If you have questions, some answers may be found in ourcontributing guidelines.

@github-actionsgithub-actionsbot added topic-documentation topic-NamedArrayLightweight version of Variable labelsJul 13, 2025
@charles-turner-1charles-turner-1 changed the titleAll works, just need to satisfy mypy and whatnot nowAdd chunks='auto' support for cftime datasetsJul 13, 2025
@jemmajeffree
Copy link

Would these changes also work for cf timedeltas or are they going to still cause problems?
I'm tempted to write a script to bash through all the ACCESS-NRI intake datastores and see if there's anything else in there that's dtype object — let me know if this would be useful, or if we should just wait for it to break later

@charles-turner-1
Copy link
Author

Would these changes also work for cf timedeltas or are they going to still cause problems? I'm tempted to write a script to bash through all the ACCESS-NRI intake datastores and see if there's anything else in there that's dtype object — let me know if this would be useful, or if we should just wait for it to break later

If you can find something thats specifically a cftimedelta and run the_contains_cftime_datetimes function on it that'd be super helpful to know whether it returns True or False.

jemmajeffree reacted with thumbs up emoji

@charles-turner-1charles-turner-1 marked this pull request as draftJuly 14, 2025 05:02
@jemmajeffree
Copy link

TLDR: don't mind me, it's not going to cause any issues

Firstly, what I thought was a cftimedelta turned out to be a numpy timedelta hanging out with a cftime
Screenshot 2025-07-14 at 5 23 31 pm
When I did manage to coerce this timedelta into cftime conventions, it just contained a floating point number of days, so I can't see anything having issues with its size

coder=xr.coding.times.CFTimedeltaCoder()result=coder.encode(oops.average_DT).load()print(result.dtype)result
Screenshot 2025-07-14 at 5 38 33 pm

@charles-turner-1
Copy link
Author

I did some prodding around yesterday and I realised this won't let us do something like

importxarrayasxrcftime_datafile="/path/to/file.nc"xr.open_dataset(cftime_datafile,chunks='auto')

yet, only stuff along the lines of

importxarrayasxrcftime_datafile="/path/to/file.nc"ds=xr.open_dataset(cftime_datafile,chunks=-1)ds=ds.chunk('auto')

I think implementing the former is going to be a bit harder, but I'm starting to clock the code structure a bit more now so I'll have a decent crack.

@dcherian
Copy link
Contributor

Why so? Are we sending"auto" in tonormalize_chunks first?

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@dcheriandcheriandcherian left review comments

Assignees
No one assigned
Labels
topic-documentationtopic-NamedArrayLightweight version of Variable
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@charles-turner-1@jemmajeffree@dcherian

[8]ページ先頭

©2009-2025 Movatter.jp