Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.4k
gh-127119: Remove check on accidental deallocation of immortal objects for free-threaded build#127120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
skirpichev commentedNov 22, 2024
Apparently, assertion is broken in tests. |
eendebakpt commentedNov 22, 2024
Yes, my mistake. Should be fixed now |
Uh oh!
There was an error while loading.Please reload this page.
Misc/NEWS.d/next/Core_and_Builtins/2024-11-22-07-58-00.gh-issue-127119.p9Yv4U.rst OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
…e-127119.p9Yv4U.rstCo-authored-by: Peter Bierma <zintensitydev@gmail.com>
ZeroIntensity left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
LGTM
bedevere-bot commentedNov 22, 2024
🤖 New build scheduled with the buildbot fleet by@ZeroIntensity for commit4dc97c9 🤖 If you want to schedule another build, you need to add the🔨 test-with-refleak-buildbots label again. |
CI is failing, because apparently you can build with assertions but not with debug enabled (that's what I get for prematurely approving, ha).
Uh oh!
There was an error while loading.Please reload this page.
ZeroIntensity left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
LGTM (again)!
gpshead left a comment• edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
this changes from silently re-immortalizing the small ints to actually letting them be deallocated in non-debug builds. in theory this is supposed to be fine, but how confident are we of (buggy) reference counting code not relying in the former behavior?
(I'm not blocking this PR, I think the change is acceptable, just want that pondered...)
ZeroIntensity commentedNov 24, 2024
I thought about that too, but Ithink issues around immortal objects being buggy was fixed byGH-125251 (at least, I thought that was implied by the removal of |
eendebakpt commentedNov 24, 2024
The possibility of de-allocating immortal objects is described in [PEP 683, sectionAccidental De-Immortalizing](https://peps.python.org/pep-0683/#accidental-de-immortalizing). Also see But if I understand the correctly the accidental-de-immortalizing cannot happen on: 64-bit systems, or with the free-threading build, or when And yes, this should be double checked! |
eendebakpt commentedNov 24, 2024
Results on a microbenchmark: I expect no overall improvement on the pyperformance benchmark. Benchmark script |
ZeroIntensity left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This is now incorrect :(
Objects/longobject.c Outdated
| #ifdefPy_LIMITED_API | ||
| #ifndefPy_GIL_DISABLED |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I'm not sure it's possible for us to know when we're using the limited API.Py_LIMITED_API is something that affectsPython.h at the compile-time of an extension module, but this happens during the compilation of CPython, so it will never be true.
markshannon commentedDec 4, 2024
As an alternative to removing the check altogether we could make it cheaper by using the reserved bithttps://github.com/python/cpython/blob/main/Include/cpython/longintrepr.h#L79 as a marker for immortal/small ints. |
eendebakpt commentedDec 4, 2024
@markshannon Thanks for the idea! I think it works and would not be hard to implement, but I am a bit hesitant to use a reserved bit for a small optimization like this. I created an alternative PR with a faster check based on your idea though |
markshannon commentedDec 5, 2024
That bit is reserved for exactly this use case, so it is fine to use it. |
eendebakpt commentedJan 28, 2025
Closing in favor of#127620 |
Uh oh!
There was an error while loading.Please reload this page.
(Description was updated) The python small ints are immortal with a fixed reference count (also some other types). However, old extension modules can still modify the reference count and end up deallocating these objects.
This should not be an issue on 64-bit builds (even when accidentically changing the reference count it will take a very long time for an object to reach refcount zero) or the free-threaded build (which used a different reference counts, so has no legacy extensions modules)
In this PR we remove the check on accidental deallocation for the free-threading build. This has performance impact for the long type (where the
tp_deallocis actually called often). For the other types we remove it to be consistent, but it should have no performance impact.Also see: