Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.3k
Closed
Description
Bug report
Bug description:
Sub interpreters have a check (_PyImport_CheckSubinterpIncompatibleExtensionAllowed) for extensions which aren't compatible.
One example of an incompatible extension isreadline. It has a global state and shouldn't be imported from a sub interpreter. The issue is that it can be imported dynamically and the crash happens in the module init mechanism, but the check isafter (https://github.com/python/cpython/blob/main/Python/importdl.c#L208) meaning it doesn't have the chance to stop the import before the segmentation fault.
I discovered this by writing a simple test harness that goes through the Python test suite and tries to run each test in a sub interpreter. It segfaults on a number of test suites (test_builtin is the first one)
fromtest.libregrtest.findtestsimportfindtestsimport_xxsubinterpretersasinterpreters# Get a list of teststest_names=findtests()skip_tests= [# "test_builtin" # Crashes]defrun_test():importunittesttest_cases=unittest.defaultTestLoader.loadTestsFromName(f"test.{test_name}")reasons=""pass_count=0fail_count=0forcaseintest_cases:r=unittest.result.TestResult()case.run(r)ifr.wasSuccessful():pass_count+=r.testsRunelse:forfailedcase,reasoninr.failures:reasons+="---------------------------------------------------------------\n"reasons+=f"Test case{failedcase} failed:\n"reasons+=reasonreasons+="\n---------------------------------------------------------------\n"fail_count+=1forfailedcase,reasoninr.errors:reasons+= ("---------------------------------------------------------------\n" )reasons+=f"Test case{failedcase} failed with errors:\n"reasons+=reasonreasons+="\n---------------------------------------------------------------\n"fail_count+=1interp=interpreters.create()fortestintest_names:# Run the test suiteiftestinskip_tests:print(f"Skipping test{test}")continueprint(f"Running test{test}")try:result=interpreters.run_func(interp,run_test,shared={"test_name":test})exceptExceptionase:print(f"Test{test} failed with exception{e}")continue
This crashes during thetest_builtin suite and any others which dynamically load the readline module.
>./python.exe -X dev test_in_interp.py....Running test test_bufioRunning test test_builtinFatal Python error: Segmentation faultCurrent thread 0x00007ff857472700 (most recent call first): File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed File "<frozen importlib._bootstrap_external>", line 1304 in create_module File "<frozen importlib._bootstrap>", line 813 in module_from_spec File "<frozen importlib._bootstrap>", line 915 in _load_unlocked File "<frozen importlib._bootstrap>", line 1325 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1354 in _find_and_load File "/Users/anthonyshaw/projects/cpython/Lib/pdb.py", line 239 in __init__ File "/Users/anthonyshaw/projects/cpython/Lib/doctest.py", line 386 in __init__ File "/Users/anthonyshaw/projects/cpython/Lib/doctest.py", line 1527 in run File "/Users/anthonyshaw/projects/cpython/Lib/doctest.py", line 2261 in runTest File "/Users/anthonyshaw/projects/cpython/Lib/unittest/case.py", line 589 in _callTestMethod File "/Users/anthonyshaw/projects/cpython/Lib/unittest/case.py", line 636 in run File "/Users/anthonyshaw/projects/cpython/Lib/unittest/case.py", line 692 in __call__ File "/Users/anthonyshaw/projects/cpython/Lib/unittest/suite.py", line 122 in run File "/Users/anthonyshaw/projects/cpython/test_in_interp.py", line 19 in run_testExtension modules: _testcapi, _xxsubinterpreters (total: 2)zsh: segmentation fault ./python.exe -X dev test_in_interp.py
Stack trace:
python.exe!Py_TYPE (/Users/anthonyshaw/projects/cpython/Include/object.h:297)python.exe!Py_IS_TYPE (/Users/anthonyshaw/projects/cpython/Include/object.h:330)python.exe!PyObject_TypeCheck (/Users/anthonyshaw/projects/cpython/Include/object.h:487)python.exe!PyModule_GetState (/Users/anthonyshaw/projects/cpython/Objects/moduleobject.c:608)readline.cpython-313td-darwin.so!on_startup_hook (/Users/anthonyshaw/projects/cpython/Modules/readline.c:1029)libedit.3.dylib!rl_initialize (Unknown Source:0)readline.cpython-313td-darwin.so!setup_readline (/Users/anthonyshaw/projects/cpython/Modules/readline.c:1219)readline.cpython-313td-darwin.so!PyInit_readline (/Users/anthonyshaw/projects/cpython/Modules/readline.c:1515)python.exe!_PyImport_LoadDynamicModuleWithSpec (/Users/anthonyshaw/projects/cpython/Python/importdl.c:170)python.exe!_imp_create_dynamic_impl (/Users/anthonyshaw/projects/cpython/Python/import.c:3750)python.exe!_imp_create_dynamic (/Users/anthonyshaw/projects/cpython/Python/clinic/import.c.h:485)python.exe!cfunction_vectorcall_FASTCALL (/Users/anthonyshaw/projects/cpython/Objects/methodobject.c:425)python.exe!_PyVectorcall_Call (/Users/anthonyshaw/projects/cpython/Objects/call.c:273)python.exe!_PyObject_Call (/Users/anthonyshaw/projects/cpython/Objects/call.c:348)python.exe!PyObject_Call (/Users/anthonyshaw/projects/cpython/Objects/call.c:373)python.exe!_PyEval_EvalFrameDefault (/Users/anthonyshaw/projects/cpython/Python/generated_cases.c.h:5382)python.exe!_PyEval_EvalFrame (/Users/anthonyshaw/projects/cpython/Include/internal/pycore_ceval.h:115)python.exe!_PyEval_Vector (/Users/anthonyshaw/projects/cpython/Python/ceval.c:1783)python.exe!_PyFunction_Vectorcall (Unknown Source:0)python.exe!_PyObject_VectorcallTstate (/Users/anthonyshaw/projects/cpython/Include/internal/pycore_call.h:168)CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
Metadata
Metadata
Assignees
Projects
Status
Done