Hmm, seeing this I'm really not sure it is worth breaking every package that implemented__array_wrap__ for something so relatively small... In the end, most classes could check whetherndim==0 and then do something based on that.

Another idea, that does not break the signature (but might well break implementations), is to pass on to__array_wrap__ what numpy would return, i.e., an array or a scalar.

But there also remains something to be said for just more generally using array scalars, and let that be the API break. It might just make code simpler rather than more complex...

p.s. Should stress that I haven't thought enough about this! I should really look more at yourtry-0d-preservation branch...

It would be nice not to require this, but...

In the end, most classes could check whether ndim==0 and then do something based on that.

No because 0-D doesn't imply that NumPy should be returning a scalar, after all that is what annoys us all the time! I would like to be able to decide thatnp.add(1, 1) should maybe a scalar, butnp.add(np.array(1), 1) should be an array. Orarr1d.sum() vs.arr1d.sum(-1) maybe/probably.

It doesn't matter much which cases are which, so long there is no absolute uniform 0-d is always array/scalar...

is to pass on to__array_wrap__ what numpy would return, i.e., an array or a scalar.

Might work out mostlyuntildtype=object hits and all is lost, since you need at leastobj.view(my_subclass) to work, and it cannot. And it isn't even really reasonable to check forisinstance(obj, ndarray)...

Which was, why I thought I would just callresult[()] instead, but that also doesn't work because we don't know that the result defines such indexing semantics.

So, I would love a different idea, but the only thing I have left is piggy-backing oncontext or try/except indefinitely. But I really dislike try/except indefinitely, so that would leave adding one more try/except branch as the current one breaks subclasses that rely oncontext (although I hope beyond masked arrays few exist, and everyone who might want to use it moves to__array_ufunc__).

FWIW, I have updated thetry-0d-preservation branch with these (and the array-converter) changes. There are some tests still failing, but the last commit shows that it is not impossible anymore to return scalars for scalar inputs and arrays otherwise.
(With the admitted caveat in the strange case where e.g.dtype="i,i" means that a tuple is considered a scalar rather, when it normally is not.)

Which was, why I thought I would just call result[()] instead, but that also doesn't work because we don't know that the result defines such indexing semantics.

Maybe we should still go with this? I think it may cause less breakage than a new argument...

Though perhaps I'm overly worried; not sure how many projects in fact have an__array_wrap__ to start with...

Let's say this, we currently have two possible paths:

1. Functions which always return an array
2. Functions which return a scalar if the result is 0-D.

And__array_wrap__ cannot possibly match that, because even if it relies oncontext being set for the second, it will miss reductions.

So to some degree, I really want to add the ability to pick the two behaviors consciously for NumPy. Now, you could just not tell__array_wrap__ about it as it rarely needs to know in practice (most implementations are already happy to just ignore it).

I think it may cause less breakage than a new argument...

Well, in the sense of requiring less code change? Yes, for sure. In the sense of braking thingsdisregarding the deprecation warning, not sure? Since I think some things like pandas Series will run intoresult[()] not working.

Though perhaps I'm overly worried; not sure how many projects in fact have an__array_wrap__ to start with...

Right, and if I add another step which tries to passarr, context whenarr, context, return_scalar fails, then this is aDeprecationWarning and will not fail straight away (albeit add a bunch of overhead).

I don't have a strong opinion, it seems tedious, but OTOH, it seems bound to be annoying for someone to not be able to match NumPy arrays (e.g.mmap is such a case).

@mhvk sorry, need to come back to this, I think it is pretty important (if just to push the followup ofarray_coverter(), whichis important to fixup NEP 50 niche case bugs in the Pytho API).

We had discussed this in a meeting a while ago, and I don't think anyone had serious concerns with adding it. But I do value your opinion more than most when it comes to__array_function__. What I will say is that this is just a deprecation, so yes, everyone must update, but end-users wouldn't see it immediately (although it's slow). It could be aPendingDeprecationWarning, but that just hides it even more with no gain I think. I can will add a try for__array_function__(arr, context), so that it is truly just a deprecation (even if it could go through 2 trys before succeeding).

To me, the new kwarg still just seems easier than any other polymorphism to try to guess this without it, and it is only some libraries that need to adapt.

Nevermind, I siwtched to not do force-wrap which I think leaves the reduction result unchanged. While I would like to align them, I need this in more for the other PR then delay thinking about it.

gh-25635 was a good reason to chain exceptions. Because this wasalways terrible (really it got no worse, except now you have a chance of getting to the real error). It shows nicely how that try/except can go wrong if it isn't very specific enough.

In [4]: np.sqrt(np.ma.masked)---------------------------------------------------------------------------TypeError                                 Traceback (most recent call last)File ~/forks/numpy/build-install/usr/lib/python3.10/site-packages/numpy/ma/core.py:6686, in MaskedConstant.__array_wrap__(self, obj, context, return_scalar)   6685 def __array_wrap__(self, obj, context=None, return_scalar=False):-> 6686     return self.view(MaskedArray).__array_wrap__(obj, context)File ~/forks/numpy/build-install/usr/lib/python3.10/site-packages/numpy/ma/core.py:3131, in MaskedArray.__array_wrap__(self, obj, context, return_scalar)   3129     fill_value = self.fill_value-> 3131 np.copyto(result, fill_value, where=d)   3133 # Update the maskTypeError: Cannot cast scalar from dtype('float64') to dtype('bool') according to the rule 'safe'The above exception was the direct cause of the following exception:DeprecationWarning                        Traceback (most recent call last)Cell In[4], line 1----> 1 np.sqrt(np.ma.masked)DeprecationWarning: __array_wrap__ must accept context and return_scalar arguments (positionally) in the future. (Deprecated NumPy 2.0)

Thanks Marten, let me squash merge this then, since I want to rebase theother PR on this work...

Afterall, while I think it fixes a bunch of issues (at least mid-term) and opens avenues, it was really mostly a split-out from the other PR.

Nice, on to the helper! (ping me when rebased - that one we went over a lot already so should be easy to do a final round of review).

I thought numpy 2 would be out by now. Wasn't expecting breaking changes even now...

I thought numpy 2 would be out by now. Wasn't expecting breaking changes even now...

I wish :(, although I hope it should be mostly good. You will certainly also noticegh-25168 (more than this).