Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-116909: fix data race with versions in typeobject#134651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
duaneg wants to merge5 commits intopython:main
base:main
Choose a base branch
Loading
fromduaneg:gh-116909

Conversation

duaneg
Copy link
Contributor

@duanegduaneg commentedMay 25, 2025
edited by bedevere-appbot
Loading

Global state_PyRuntime.types.next_version_tag is being accessed without synchronization or atomics. This could potentially result in the same version being used in two different type objects, incrementing past the maximum limit, and the usual non-atomic memory access issues.

Fix this by using atomics to ensure the version is accessed and updated in a race-free manner, while also ensuring it is never incremented past the expected (maximum + 1).

Note there is a theoretical change in behaviour with the second use, for static builtin types, if their versions exceed the maximum, with assertions disabled: previously they would have continued incrementing past the maximum and using the increasing version number; now they will all use the maximum. It might be better to replace theassert with anabort(): I assume we would be crashing shortly if we continue after this, both before and after this change.

Global state `_PyRuntime.types.next_version_tag` is being accessed withoutsynchronization or atomics. This could potentially result in the same versionbeing used in two different type objects, incrementing past the maximum limit,and the usual non-atomic memory access issues.Fix this by using atomics to ensure the version is accessed and updated in arace-free manner, while also ensuring it is never incremented past the expected(maximum + 1).Note there is a theoretical change in behaviour with the second use, for staticbuiltin types, if their versions exceed the maximum, with assertions disabled:previously they would have continued incrementing past the maximum and usingthe increasing version number; now they will all use the maximum. It might bebetter to replace the `assert` with an `abort()`: I assume we would be crashingshortly if we continue after this, both before and after this change.
Copy link
Member

@ZeroIntensityZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for doing this. I'm not sure we're able to skip the subinterpreter test yet, because there are still races when doing non-atomic stores of the version on static types. I've looked into fixing those myself, and it'ssignficantly more complex.

@duaneg
Copy link
ContributorAuthor

Thanks for doing this

My pleasure!

I'm not sure we're able to skip the subinterpreter test yet, because there are still races when doing non-atomic stores of the version on static types. I've looked into fixing those myself, and it'ssignficantly more complex.

Hmm, yes. I had tested running it with thread sanitizer enabled, with-and-without the GIL, and didn't trigger any data race reports. However, adding a new test specifically to try and exercise static type initialization indeed shows abunch of problems remain. Not only with version, but various other accesses (most oftentp_flags).

I suppose those problems may also be hit with the existing test, depending on timing. I'll drop re-enabling that test for now. I might come back and try and fix some of these issues separately, if I have time and no-one else gets there first.

For reference, the test I used:

@requires_subinterpreters@threading_helper.requires_working_threading()deftest_static_type_initialization(self):# This is specifically testing static type initialization thread safety# and is intended to be run with the thread-santizer enableddefinit(barrier):interpid=_interpreters.create()barrier.wait()try:_interpreters.run_string(interpid,"import datetime")finally:_interpreters.destroy(interpid)N=3ITERATIONS=10barrier=threading.Barrier(N)for_inrange(ITERATIONS):threads= [threading.Thread(target=init,args=(barrier,))for_inrange(N)]withthreading_helper.start_threads(threads):pass

I'm sure using subinterpreters is not strictly required, but they seemed like an easy way to reliably run static type initialisation. Note that if assertions are enabled this test immediately fails:

Objects/typeobject.c:208: managed_static_type_state_init: Assertion `!managed_static_type_index_is_set(self)' failed.

It definitely looks like this is all very racy.

Copy link
Member

@ZeroIntensityZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Mostly LGTM

…e-116909.FGbNKx.rstCo-authored-by: Peter Bierma <zintensitydev@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@ZeroIntensityZeroIntensityZeroIntensity left review comments

@markshannonmarkshannonAwaiting requested review from markshannonmarkshannon is a code owner

Assignees
No one assigned
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

2 participants
@duaneg@ZeroIntensity

[8]ページ先頭

©2009-2025 Movatter.jp