Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-126907: Lock theatexit state on the free-threaded build#126908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed

Conversation

@ZeroIntensity
Copy link
Member

@ZeroIntensityZeroIntensity commentedNov 16, 2024
edited by bedevere-appbot
Loading

Copy link
Member

@picnixzpicnixz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Quick review while waiting for my train

@kumaraditya303kumaraditya303 self-requested a reviewDecember 1, 2024 06:38
Copy link
Member

@vstinnervstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Overall, IMO the approach is valid. Here is a first review.

Copy link
Member

@vstinnervstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM.

@colesbury@mpage: Do you want want to review this change?

Copy link
Contributor

@kumaraditya303kumaraditya303 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think it's easy to introduce reentrancy issues in this code, how about using a critical section instead of mutex?

Copy link
Contributor

@colesburycolesbury left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for taking this on Peter. Like@kumaraditya303 suggested, I think using critical sections will simplify the code and avoid some re-entrancy issues. Having to explicitly unlock the mutex inside some functions is a sign that the code should be refactored:

The low-level callbacks (ll_callbacks) and high-level Python callbacks are pretty different:

  • ThePyMutex for modifyingll_callbacks is fine
  • Use a critical section for protectingstate->callbacks. Ideally, the acquisition happens at the module boundaries. Use argument clinic and@critical_section for the module methods.
  • I suspect a lot of the code would be simplified ifstate->callbacks were a Python list object

ZeroIntensity reacted with thumbs up emoji
Comment on lines +346 to +347
_PyAtExit_LOCK(state);
if (state->callbacks[i]==NULL)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The explicit unlocking means that this still has re-entrancy and concurrency issues if the callback list is modified during the comparison.callbacks[i] may be adifferent callback.

Using critical sections doesn't completely avoid the problem if__eq__ is particularly weird, but it avoids it for the common case of identity comparison and the issues are limited to the cases where the GIL-enabled build has issues as well.

@ZeroIntensity
Copy link
MemberAuthor

Ok, I'll switch this over to critical sections. For now, I'd rather just ignorell_callbacks until I fixPyUnstable_AtExit (gh-127793).

I suspect a lot of the code would be simplified if state->callbacks were a Python list object

It could, but it also might be a PITA because we'd need to introduce capsules into the mix. I'll play around with it and see how it looks.

@colesbury
Copy link
Contributor

It could, but it also might be a PITA because we'd need to introduce capsules into the mix. I'll play around with it and see how it looks.

I don't think you need capsules, which I agree would be a pain. I think you can use a tuple of three items instead ofatexit_py_callback and replaceatexit_py_callback **callbacks with a list of those tuples.

Most of whatatexit does with thePython callbacks is essentially list manipulation and we already have thread-safe implementations for PyListObject. You might need to exposelist_remove as an internal-only function though.

The Python callbacks are executed while other threads are still running (and can possibly modify the callbacks). Using a PyListObject makes it easier to make a local copy of the callbacks before iterating over them, which avoids some re-entrancy and thread-safety issues inatexit_callfuncs that aren't limited to the free threading build.

@colesbury
Copy link
Contributor

To be clear, I am only suggesting a PyListObject instead ofatexit_py_callback **callbacks (the high-level Python callbacks) and not instead ofatexit_callback *ll_callbacks (the low-level C callbacks).

ZeroIntensity reacted with thumbs up emoji

@ZeroIntensity
Copy link
MemberAuthor

I was just thinking about using a list because it has a nicer interface, not for thread safety.

The Python callbacks are executed while other threads are still running (and can possibly modify the callbacks).

They are? That's not good. The two places that execute atexit callbacks (apart from manually doing it) are

_PyAtExit_Call(tstate->interp);
and
_PyAtExit_Call(tstate->interp);

Both of which aresupposed to happen when no threads are left. I guess daemon and C threads could screw things up, but I'm willing to write that off as a "play stupid games, win stupid prizes."

@ZeroIntensity
Copy link
MemberAuthor

Marking asDO-NOT-MERGE while I refactor this. I'll (hopefully) have this ready by the end of the week.

@vstinner
Copy link
Member

You might create a new PR if you switch to critical sections, so we can compare the two approaches, and the PR history is easier to follow.

ZeroIntensity reacted with thumbs up emoji

@ZeroIntensity
Copy link
MemberAuthor

Closing now that#127935 got merged. Thanks to everyone that reviewed!

vstinner reacted with thumbs up emoji

@ZeroIntensityZeroIntensity deleted the atexit-threadsafety branchDecember 16, 2024 19:32
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@colesburycolesburycolesbury left review comments

@picnixzpicnixzpicnixz left review comments

@kumaraditya303kumaraditya303kumaraditya303 left review comments

@vstinnervstinnervstinner approved these changes

@ericsnowcurrentlyericsnowcurrentlyAwaiting requested review from ericsnowcurrentlyericsnowcurrently is a code owner

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ZeroIntensity@colesbury@vstinner@picnixz@kumaraditya303

[8]ページ先頭

©2009-2025 Movatter.jp