Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32.4k
gh-136421: Load_datetime
static types during interpreter initialization#136583
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
I have no idea if this will fix the Windows build, but let's hope.
My concern is unnecessarily losing portability between the builtin module and the non-builtin one. This PR avoids a crash not by locking but forcing the main interpreter initialize the PyMODINIT_FUNCPyInit__datetime(void){+ if (PyDateTime_DateType.tp_subclasses == NULL) {+ PyInterpreterState *interp = PyInterpreterState_Get();+ // main interp currently checks a module before subinterp imports+ assert(_Py_IsMainInterpreter(interp));+ init_static_types(interp, 0);+ } return PyModuleDef_Init(&datetimemodule);} |
Hmm, that feels really hacky. I wouldn't call the movement of |
The crash can be avoided even on pure Python level, and managed staticextsension types are already special only for
Not sure about disabling. On Windows, |
Right, there are several ways to fix this thatwork, but I'm trying to find the fix that is the mostcorrect. Relying on implementation details like
We have to find a balance between correctness and simplicity, and I'm not yet sure which that is. |
This is at least a feature change more than a bug fix. |
Yeah, you're right. I'd be fine with a bandaid fix for 3.14 if you want to make a PR. The |
Still rough example, but checking During the |
|
A small concern is the startup time. Can you do a micro performance benchmark to measure the time cost of |
#136620 is ready for review.
I haven't come up with how to measure so far. |
status = _PyDateTime_InitTypes(interp); | ||
if (_PyStatus_EXCEPTION(status)) { | ||
return status; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
status=_PyDateTime_InitTypes(interp); | |
if (_PyStatus_EXCEPTION(status)) { | |
returnstatus; | |
} | |
if (!_Py_IsMainInterpreter(interp)) { | |
status=_PyDateTime_InitTypes(interp); | |
if (_PyStatus_EXCEPTION(status)) { | |
returnstatus; | |
} | |
} |
Reply from#136620 (comment)
Can you runtest_concurrent_initialization()
with this change? Based on the crashes that come from the change, I said this PR and my example are "almost equivalent." I'm not sure right now what this PR ensures.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hm, what are you trying to achieve here? This will just break the types for the main interpreter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This will just break the types for the main interpreter.
Note thattest_concurrent_initialization()
does not load the_datetime
in the main inter interpreter at all.
Correction: Run thescript of the test without runningtest_datetime
.
Uh oh!
There was an error while loading.Please reload this page.
_datetime
is a special module, because it's the only non-builtin C extension that contains static types. As such, it would initialize static types in the module's execution function, which can run concurrently. Since static type initialization is not thread-safe, this caused crashes. This fixes it by moving the initialization of_datetime
's static types to interpreter startup (where all other static types are initialized), which is already properly protected through other locks._datetime
in sub-interpreters in the same time may crash the process #136421