- Notifications
You must be signed in to change notification settings - Fork54
Intent to adapt Array API in Python-Blosc2#968
-
Hi! ThePython-Blosc2 project is trying to adapt to the Array API specs. We have started work here:Blosc/python-blosc2@main...array-api The Array API has been in our radar at least since 2023, when I attended to the Aaron talk in SciPy conference, but probably I forgot the details and was naively thinking that just adapting to name convention was going to be enough. During recent EuroSciPy conference, it was clear that adapting to Array API is much more than that; thanks to@seberg and@lucascolley for making that clearer and for their help in our initial effort! In case anyone has suggestions, or is willing to help us in this attempt, please shout! |
BetaWas this translation helpful?Give feedback.
All reactions
🎉 1🚀 5
Replies: 4 comments 15 replies
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
-
Hi, which is equivalent to ? |
BetaWas this translation helpful?Give feedback.
All reactions
-
If all the functions in the array namespace accept NumPy arrays as input, this does seem like a minor issue. I guess there are some things which will just be wrong though, like Would it be an option to return a subclass of |
BetaWas this translation helpful?Give feedback.
All reactions
❤️ 1
-
That is also quite a good idea thanks very much. I have opened a ticketBlosc/python-blosc2#464 (comment). Will try to work on it in the medium term and focus on the rest of the array-api suite for the moment. |
BetaWas this translation helpful?Give feedback.
All reactions
👍 1
-
Please don't hesitate reaching out, here or in the |
BetaWas this translation helpful?Give feedback.
All reactions
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
-
The problem with this is mainly that we don't allow "promotion", i.e. no way to mix different array types in (The question is whether you want users to mix NumPy and blosc arrays, because if you do, users may run into this issue whether the subclass exists or not. And the only fix is to allow array-namespace to figure things out. That isn't hard, Array API just didn't want to add the complexity.) EDIT: If you are happy with the subclass, fine, it's avery simple one. But it leaves me a bit unsatisfied because I think it is an inelegant solution for what is an Array API issue/limitation. |
BetaWas this translation helpful?Give feedback.
All reactions
-
Yeah, we have discussed that internally, and I am not a big fan of the subclass either, precisely because I feel it inelegant too, but also because it adds another class that the user has to know about. For the time being we will just skip the tests that are related with this. Perhaps in the future we can come with a solution that satisfies most of us. |
BetaWas this translation helpful?Give feedback.
All reactions
👍 1
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
-
Hi, I have another question - I suppose it is perhaps related to the test suite but it's not really an issue per se. When executing the tests for |
BetaWas this translation helpful?Give feedback.
All reactions
-
At a guess, that's hypothsis generating a too-large array to test. Note that assertions in Try passing |
BetaWas this translation helpful?Give feedback.
All reactions
👍 1
-
Another thing that I'd like to bring on the table is the use of an optional It would be cool if Array API may adopt the same convention and allow an optional |
BetaWas this translation helpful?Give feedback.
All reactions
-
I imagine this is just a question of whether current adoption is widespread enough to justify requiring it from all compliant implementations. |
BetaWas this translation helpful?Give feedback.
All reactions
👍 1
-
At the moment, libraries, such as NumPy, JAX, Torch, and Dask, don't support a |
BetaWas this translation helpful?Give feedback.
All reactions
-
Could blosc evaluate |
BetaWas this translation helpful?Give feedback.
All reactions
-
Ok, that looks like a good idea, and performance wise, the difference is not that much: |
BetaWas this translation helpful?Give feedback.
All reactions
👍 1
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
-
great! I guess this is why the standard includes |
BetaWas this translation helpful?Give feedback.
All reactions
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
-
Ok, so with Blosc2 3.9 series we did a big push towards array API compatibility:https://github.com/Blosc/python-blosc2/releases Hopefully we can get some funding for making more progress in the future, but what we have now is a pretty good basic compliance. Thanks for your help so far! |
BetaWas this translation helpful?Give feedback.
All reactions
👍 3🚀 1