Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

BUG: Silencing errors of filled_value's casting in array_finalize in MaskedArray#27596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
fengluoqiuwu wants to merge8 commits intonumpy:main
base:main
Choose a base branch
Loading
fromfengluoqiuwu:fix-bug-27165

Conversation

fengluoqiuwu
Copy link
Contributor

@fengluoqiuwufengluoqiuwu commentedOct 19, 2024
edited
Loading

Silencing errors related to the casting offill_value in__array_finalize__ ofMaskedArray, and using a default value if the originalfill_value cannot be cast even with the method_check_fill_value.

While this approach may introduce some level of risk, it is reasonable to assume that if users set thefill_value themselves, they are aware that it may lead to undefined behavior with specific type casting. It is unfortunate that the casting ofmasked_array types is hindered simply because we assignedfill_value with the default value indirectly. This not only causes issues with methods that usedtype= directly, but also impedes any calls toufunc that return a dtype different from the original dtype (since calls to theufunc won't be applied to the storedfill_value), exacerbating the situation.

I believe this is the most effective way to address these issues, rather than designing a new ufunc formasked_array or marking whether the value was set by the user or not.
fixes#27165

@melissawmmelissawm added the component: numpy.mamasked arrays labelOct 24, 2024
@seberg
Copy link
Member

@fengluoqiuwu two things here:

  1. It would be nice to explain where theTypeError comes from. As a reviewer if I accept that catching theTypeError may be the best thing (probably alsoOverflowError?) the next thing will be to ask whether this is the right point.
  2. We really can't merge even an obvious bug fix without a test and a test is also very helpful for reviewers.

@fengluoqiuwu
Copy link
ContributorAuthor

It would be nice to explain where theTypeError comes from.

TheTypeError is raised by the_check_fill_value method, which is converted various errors we want to catch, such asOverflowError, intoTypeError.

We really can't merge even an obvious bug fix without a test and a test is also very helpful for reviewers.

I will add it later.

@seberg
Copy link
Member

Thanks! Looking a bit closer, I suspect this is right and we could probably just do it. We don't know that the dtype matches, and we have to just fall back to the default when the "inherited" fill-value fails (this is part of__array_finalize__).

The reason I ask about where it comes from is that it may better to handle this in_check_fill_value itself. I see a similar branch in one place incompare where it catches(TypeError, ValueError).

In this case, I actually like if we could audit/clean up things a bit more. That probably means pushing this logic into_check_fill_value but adding afallback=False argument to it.
(From which I hope two things. First, make it more obvious why the try/except is there. And second, make sure we add this logic in all places where it should be used and not just the one where it happened to be noticed.)

I think in most places probably we should fall back toNone. But when a user doesarr.fill_value = fill_value, I don't think we should (it is the users job to pass a reasonable fill value).

(It is unfortunate, that this was always weird, but the integer casting is now floating up these issues...)

@fengluoqiuwu
Copy link
ContributorAuthor

I think in most places probably we should fall back toNone.

I think using the default value is better. In the comment for_check_fill_value, it says that it will only return anndarray, so the return value won’t be checked before use. Additionally, in_comparison, it uses the default value, which is the same situation as this one.

seberg added a commit to seberg/numpy that referenced this pull requestNov 20, 2024
This is the most minimal fix I could think off... If we have uintsstill use the out-of-bounds value, but make it uint.  This is becausethe code uses copyto with the default same-kind casting in places...This doesn't remove all the other quirks, and surely, it is a behaviorchange in some awkward situations (since you got a uint, someone mightmix the newly uint value with an integer and get float64 results, etc.)But... it is by far the most minimal fix I could think of after a longertime.A more thorough fix may be to just always store the exact dtype, butit would propagate differently e.g. when casting.  A start for thisis currently innumpygh-27596.Closesnumpygh-27269,numpygh-27580
seberg added a commit to seberg/numpy that referenced this pull requestNov 20, 2024
This is the most minimal fix I could think off... If we have uintsstill use the out-of-bounds value, but make it uint.  This is becausethe code uses copyto with the default same-kind casting in places...This doesn't remove all the other quirks, and surely, it is a behaviorchange in some awkward situations (since you got a uint, someone mightmix the newly uint value with an integer and get float64 results, etc.)But... it is by far the most minimal fix I could think of after a longertime.A more thorough fix may be to just always store the exact dtype, butit would propagate differently e.g. when casting.  A start for thisis currently innumpygh-27596.Closesnumpygh-27269,numpygh-27580
ArvidJB pushed a commit to ArvidJB/numpy that referenced this pull requestJan 8, 2025
This is the most minimal fix I could think off... If we have uintsstill use the out-of-bounds value, but make it uint.  This is becausethe code uses copyto with the default same-kind casting in places...This doesn't remove all the other quirks, and surely, it is a behaviorchange in some awkward situations (since you got a uint, someone mightmix the newly uint value with an integer and get float64 results, etc.)But... it is by far the most minimal fix I could think of after a longertime.A more thorough fix may be to just always store the exact dtype, butit would propagate differently e.g. when casting.  A start for thisis currently innumpygh-27596.Closesnumpygh-27269,numpygh-27580
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Projects
Status: Awaiting a code review
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

BUG: inconsistent fill_value casting for masked arrays
3 participants
@fengluoqiuwu@seberg@melissawm

[8]ページ先頭

©2009-2025 Movatter.jp