Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-98354: Add unicode check for 'name' attribute in _imp_create_builtin#98412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
miss-islington merged 7 commits intopython:mainfromchgnrdv:_imp_create_builtin-check-name-for-unicode
Oct 20, 2022
Merged

gh-98354: Add unicode check for 'name' attribute in _imp_create_builtin#98412

miss-islington merged 7 commits intopython:mainfromchgnrdv:_imp_create_builtin-check-name-for-unicode
Oct 20, 2022

Conversation

chgnrdv
Copy link
Contributor

@chgnrdvchgnrdv commentedOct 18, 2022•
edited by miss-islington
Loading

_imp_create_builtin crashes if 'name' attribute of 'spec' argument is not a 'str' instance. This commit adds appropriate check.
Copy link
Member

@JelleZijlstraJelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for contributing a fix!

Agree that this needs a new test. Also, there should be a NEWS entry.

Your check currently still allows subclasses ofstr. That's probably fine, but worth an additional test.

@ericsnowcurrently
Copy link
Member

The fix looks good but I agree a test and NEWS entry are warranted.

We'll also want to backport the fix to 3.10 and 3.11.

Copy link
Member

@ericsnowcurrentlyericsnowcurrently left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM

@miss-islingtonmiss-islington merged commit1f369ad intopython:mainOct 20, 2022
@chgnrdvchgnrdv deleted the _imp_create_builtin-check-name-for-unicode branchOctober 20, 2022 13:42
carljm added a commit to carljm/cpython that referenced this pull requestOct 20, 2022
* main: (40 commits)pythongh-98461: Fix source location in comprehensions bytecode (pythonGH-98464)pythongh-98421: Clean Up PyObject_Print (pythonGH-98422)pythongh-98360: multiprocessing now spawns children on Windows with correct argv[0] in virtual environments (pythonGH-98462)  CODEOWNERS: Become a typing code owner (python#98480)  [doc] Improve logging cookbook example. (pythonGH-98481)  Add more tkinter.Canvas tests (pythonGH-98475)pythongh-95023: Added os.setns and os.unshare functions (python#95046)pythonGH-98363: Presize the list for batched() (pythonGH-98419)pythongh-98374: Suppress ImportError for invalid query for help() command. (pythongh-98450)  typing tests: `_overload_dummy` raises `NotImplementedError`, not `RuntimeError` (python#98351)pythongh-98354: Add unicode check for 'name' attribute in _imp_create_builtin (pythonGH-98412)pythongh-98257: Make _PyEval_SetTrace() reentrant (python#98258)pythongh-98414: py.exe launcher does not use defaults for -V:company/ option (pythonGH-98460)pythongh-98417: Store int_max_str_digits on the Interpreter State (pythonGH-98418)  Doc: Remove title text from internal links (python#98409)  [doc] Refresh the venv introduction documentation, and correct the statement about VIRTUAL_ENV (pythonGH-98350)  Docs: Bump sphinx-lint and fix unbalanced inline literal markup (python#98441)pythongh-92886: Replace assertion statements in `handlers.BaseHandler` to support running with optimizations (`-O`) (pythonGH-93231)pythongh-92886: Fix tests that fail when running with optimizations (`-O`) in `_test_multiprocessing.py` (pythonGH-93233)pythongh-92886: Fix tests that fail when running with optimizations (`-O`) in `test_py_compile.py` (pythonGH-93235)  ...
@chgnrdv
Copy link
ContributorAuthor

@ericsnowcurrently@sweeneyde
Looks like import ofbuiltins module in my test (both throughimport statement andimport_helper.import_module function) leads interpreter to crash iftest_imp is running as a part of full test package (i. e. launched throughpython -m test), at least on my machine. It can be prevented by not importing any module in my test at all and just checkingbltin value's__name__ attribute, but does it deserve separate issue/PR? Documentation oftest package says that excessive imports can lead to "anomalous behaviour", is this the case?
test_imp is running OK when launched separately.

@ericsnowcurrently
Copy link
Member

Thanks for bringing this up,@chgnrdv!

Looks like import ofbuiltins module in my test (both throughimport statement andimport_helper.import_module function) leads interpreter to crash iftest_imp is running as a part of full test package (i. e. launched throughpython -m test), at least on my machine. [snip]test_imp is running OK when launched separately.

CI would have caught this if it's a consistent problem. Is this intermittent (depending on the order of tests?

It can be prevented by not importing any module in my test at all and just checkingbltin value's__name__ attribute, but does it deserve separate issue/PR? Documentation oftest package says that excessive imports can lead to "anomalous behaviour", is this the case?

It shouldn't be crashing though. Furthermore, if you are still seeing the behavior fromgh-98354 in some cases then we should fix it. I've re-opened the issue.

@chgnrdv
Copy link
ContributorAuthor

chgnrdv commentedNov 2, 2022•
edited
Loading

Is this intermittent (depending on the order of tests?

It can depend on the order of tests, since it crashes when I run the test package in default order, but passes the CI (which, AFAIK, runs tests in random order).
If I runtest_imp separately, it doesn't crash (and passes, as it should).

Furthermore, if you are still seeing the behavior from#98354 in some cases

I don't see the behaviour which I had to fix by this PR anymore and I'm sure that this issue or my fix itself can't cause any problem like this. I think that it is somehow related to how test package handles the import ofbuiltins module in my test case.

Test package output with traceback and abort message:

0:33:04 load avg: 0.83 [190/437] test_impModules/gcmodule.c:113: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too smallEnable tracemalloc to get the memory block allocation tracebackobject address  : 0x7f83fbfcb8f0object refcount : 99object type     : 0x559bf783d860object type name: moduleobject repr     : <module 'builtins' (built-in)>Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailedPython runtime state: initializedCurrent thread 0x00007f83fc323740 (most recent call first):  Garbage-collecting  File "/home/.../cpython/Lib/test/support/__init__.py", line 738 in gc_collect  File "/home/.../cpython/Lib/test/libregrtest/save_env.py", line 314 in __exit__  File "/home/.../cpython/Lib/test/libregrtest/runtest.py", line 312 in _runtest_inner2  File "/home/.../cpython/Lib/test/libregrtest/runtest.py", line 360 in _runtest_inner  File "/home/.../cpython/Lib/test/libregrtest/runtest.py", line 235 in _runtest  File "/home/.../cpython/Lib/test/libregrtest/runtest.py", line 265 in runtest  File "/home/.../cpython/Lib/test/libregrtest/main.py", line 450 in run_tests_sequential  File "/home/.../cpython/Lib/test/libregrtest/main.py", line 564 in run_tests  File "/home/.../cpython/Lib/test/libregrtest/main.py", line 742 in _main  File "/home/.../cpython/Lib/test/libregrtest/main.py", line 701 in main  File "/home/.../cpython/Lib/test/libregrtest/main.py", line 763 in main  File "/home/.../cpython/Lib/test/__main__.py", line 2 in <module>  File "/home/.../cpython/Lib/runpy.py", line 88 in _run_code  File "/home/.../cpython/Lib/runpy.py", line 198 in _run_module_as_mainExtension modules: _testcapi, _xxsubinterpreters, _testinternalcapi, _testbuffer, _testmultiphase, _ctypes_test, xxsubtype (total: 7)Aborted (core dumped)

Stack trace:

#0  0x00007f5d903b134c in __pthread_kill_implementation () from /usr/lib/libc.so.6#1  0x00007f5d903644b8 in raise () from /usr/lib/libc.so.6#2  0x00007f5d9034e534 in abort () from /usr/lib/libc.so.6#3  0x000055faf5565bf6 in fatal_error_exit (status=<optimized out>) at Python/pylifecycle.c:2629#4  0x000055faf5567236 in fatal_error (fd=2, header=header@entry=1, prefix=prefix@entry=0x55faf56427e0 <__func__.2> "_PyObject_AssertFailed",msg=msg@entry=0x55faf5642646"_PyObject_AssertFailed",status=status@entry=-1)atPython/pylifecycle.c:2810#5  0x000055faf55672a0 in _Py_FatalErrorFunc (func=func@entry=0x55faf56427e0 <__func__.2> "_PyObject_AssertFailed",msg=msg@entry=0x55faf5642646"_PyObject_AssertFailed")atPython/pylifecycle.c:2826#6  0x000055faf547f0b7 in _PyObject_AssertFailed (obj=0x7f5d8ffc7890, expr=expr@entry=0x55faf56a3268 "gc_get_refs(g) > 0",msg=msg@entry=0x55faf5642692"refcount is too small",file=file@entry=0x55faf56a30d6"Modules/gcmodule.c",line=line@entry=113,function=function@entry=0x55faf56a36e8<__func__.27>"gc_decref")atObjects/object.c:2361#7  0x000055faf559f22d in gc_decref (g=<optimized out>) at Modules/gcmodule.c:113#8  0x000055faf559f945 in visit_decref (op=0x7f5d8ffc7890, parent=0x7f5d890a3950) at Modules/gcmodule.c:459#9  0x000055faf5467b25 in dict_traverse (op=<optimized out>, visit=0x55faf559f8e2 <visit_decref>, arg=0x7f5d890a3950) at Objects/dictobject.c:3525#10 0x000055faf559df22 in subtract_refs (containers=containers@entry=0x55faf5883018 <_PyRuntime+69432>) at Modules/gcmodule.c:478#11 0x000055faf559f329 in deduce_unreachable (base=base@entry=0x55faf5883018 <_PyRuntime+69432>, unreachable=unreachable@entry=0x7ffe797ee710)atModules/gcmodule.c:1103#12 0x000055faf559feb5 in gc_collect_main (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>, generation=generation@entry=2,n_collected=n_collected@entry=0x7ffe797ee798,n_uncollectable=n_uncollectable@entry=0x7ffe797ee7a0,nofail=nofail@entry=0)atModules/gcmodule.c:1229#13 0x000055faf55a0431 in gc_collect_with_callback (tstate=0x55faf589e3e0 <_PyRuntime+180992>, generation=2) at Modules/gcmodule.c:1403#14 0x000055faf55a0513 in gc_collect_impl (module=module@entry=0x7f5d8f7707d0, generation=generation@entry=2) at Modules/gcmodule.c:1541#15 0x000055faf55a05d3 in gc_collect (module=0x7f5d8f7707d0, args=<optimized out>, args@entry=0x7f5d9062e980, nargs=nargs@entry=0,kwnames=kwnames@entry=0x0)atModules/clinic/gcmodule.c.h:139#16 0x000055faf551d845 in _PyEval_EvalFrameDefault (tstate=0x55faf589e3e0 <_PyRuntime+180992>, frame=0x7f5d9062e920, throwflag=0)atPython/ceval.c:4502#17 0x000055faf5520c0e in _PyEval_EvalFrame (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>, frame=frame@entry=0x7f5d9062e830,throwflag=throwflag@entry=0)at ./Include/internal/pycore_ceval.h:88#18 0x000055faf5520d26 in _PyEval_Vector (tstate=0x55faf589e3e0 <_PyRuntime+180992>, func=0x7f5d8f7b9ff0, locals=locals@entry=0x0,args=0x7f5d9062e7f8,argcount=<optimizedout>,kwnames=<optimizedout>)atPython/ceval.c:5826#19 0x000055faf5431288 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>)atObjects/call.c:396#20 0x000055faf5433c86 in _PyObject_VectorcallTstate (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>,callable=callable@entry=0x7f5d8f7b9ff0,args=args@entry=0x7f5d9062e7f8,nargsf=nargsf@entry=4,kwnames=kwnames@entry=0x0)at ./Include/internal/pycore_call.h:92#21 0x000055faf5433e4f in method_vectorcall (method=<optimized out>, args=0x7f5d9062e800, nargsf=<optimized out>, kwnames=0x0)atObjects/classobject.c:59#22 0x000055faf54315f9 in _PyObject_VectorcallTstate (tstate=0x55faf589e3e0 <_PyRuntime+180992>, callable=callable@entry=0x7f5d88cbf770,args=args@entry=0x7f5d9062e800,nargsf=9223372036854775811,kwnames=kwnames@entry=0x0)at ./Include/internal/pycore_call.h:92#23 0x000055faf54316c5 in PyObject_Vectorcall (callable=callable@entry=0x7f5d88cbf770, args=args@entry=0x7f5d9062e800, nargsf=<optimized out>,kwnames=kwnames@entry=0x0)atObjects/call.c:300#24 0x000055faf551c178 in _PyEval_EvalFrameDefault (tstate=0x55faf589e3e0 <_PyRuntime+180992>, frame=0x7f5d9062e778, throwflag=0)atPython/ceval.c:4206#25 0x000055faf5520c0e in _PyEval_EvalFrame (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>, frame=frame@entry=0x7f5d9062e290,throwflag=throwflag@entry=0)at ./Include/internal/pycore_ceval.h:88#26 0x000055faf5520d26 in _PyEval_Vector (tstate=0x55faf589e3e0 <_PyRuntime+180992>, func=0x7f5d8f7c8310, locals=locals@entry=0x0,args=0x7f5d8ffbe200,argcount=<optimizedout>,kwnames=<optimizedout>)atPython/ceval.c:5826#27 0x000055faf5431288 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>)atObjects/call.c:396#28 0x000055faf5433c86 in _PyObject_VectorcallTstate (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>,callable=callable@entry=0x7f5d8f7c8310,args=args@entry=0x7f5d8ffbe200,nargsf=nargsf@entry=1,kwnames=kwnames@entry=0x7f5d8fe68550)at ./Include/internal/pycore_call.h:92#29 0x000055faf5433e4f in method_vectorcall (method=<optimized out>, args=0x7f5d8ffbe208, nargsf=<optimized out>, kwnames=0x7f5d8fe68550)atObjects/classobject.c:59#30 0x000055faf5430ee1 in _PyVectorcall_Call (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>, func=0x55faf5433d0e <method_vectorcall>,callable=callable@entry=0x7f5d8fe8d1f0,tuple=tuple@entry=0x55faf5882d20<_PyRuntime+68672>,kwargs=kwargs@entry=0x7f5d8fdf9d30)atObjects/call.c:258#31 0x000055faf54311d0 in _PyObject_Call (tstate=0x55faf589e3e0 <_PyRuntime+180992>, callable=callable@entry=0x7f5d8fe8d1f0,args=args@entry=0x55faf5882d20<_PyRuntime+68672>,kwargs=kwargs@entry=0x7f5d8fdf9d30)atObjects/call.c:329#32 0x000055faf5431218 in PyObject_Call (callable=callable@entry=0x7f5d8fe8d1f0, args=args@entry=0x55faf5882d20 <_PyRuntime+68672>,kwargs=kwargs@entry=0x7f5d8fdf9d30)atObjects/call.c:356#33 0x000055faf550d9a3 in do_call_core (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>, func=func@entry=0x7f5d8fe8d1f0,callargs=callargs@entry=0x55faf5882d20<_PyRuntime+68672>,kwdict=kwdict@entry=0x7f5d8fdf9d30,use_tracing=0)atPython/ceval.c:6759#34 0x000055faf551efa4 in _PyEval_EvalFrameDefault (tstate=0x55faf589e3e0 <_PyRuntime+180992>, frame=0x7f5d9062e210, throwflag=0)atPython/ceval.c:4776#35 0x000055faf5520c0e in _PyEval_EvalFrame (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>,frame=frame@entry=0x7f5d9062e1b8,throwflag=throwflag@entry=0)at ./Include/internal/pycore_ceval.h:88#36 0x000055faf5520d26 in _PyEval_Vector (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>,func=func@entry=0x7f5d8ff83c20,locals=locals@entry=0x7f5d8fdf94f0,args=args@entry=0x0,argcount=argcount@entry=0,kwnames=kwnames@entry=0x0)atPython/ceval.c:5826#37 0x000055faf5520e3c in PyEval_EvalCode (co=co@entry=0x7f5d8fe34640, globals=globals@entry=0x7f5d8fdf94f0, locals=locals@entry=0x7f5d8fdf94f0)atPython/ceval.c:583#38 0x000055faf550721f in builtin_exec_impl (module=module@entry=0x7f5d8ffc7890, source=0x7f5d8fe34640, globals=0x7f5d8fdf94f0,locals=0x7f5d8fdf94f0,closure=0x0)atPython/bltinmodule.c:1079#39 0x000055faf550733a in builtin_exec (module=0x7f5d8ffc7890, args=<optimized out>, args@entry=0x7f5d9062e180, nargs=nargs@entry=2,kwnames=kwnames@entry=0x0)atPython/clinic/bltinmodule.c.h:543#40 0x000055faf547ab58 in cfunction_vectorcall_FASTCALL_KEYWORDS (func=0x7f5d8ffc7ef0, args=0x7f5d9062e180, nargsf=<optimized out>, kwnames=0x0)atObjects/methodobject.c:443#41 0x000055faf54315f9 in _PyObject_VectorcallTstate (tstate=0x55faf589e3e0 <_PyRuntime+180992>, callable=callable@entry=0x7f5d8ffc7ef0,args=args@entry=0x7f5d9062e180,nargsf=9223372036854775810,kwnames=kwnames@entry=0x0)at ./Include/internal/pycore_call.h:92#42 0x000055faf54316c5 in PyObject_Vectorcall (callable=callable@entry=0x7f5d8ffc7ef0, args=args@entry=0x7f5d9062e180, nargsf=<optimized out>,kwnames=kwnames@entry=0x0)atObjects/call.c:300#43 0x000055faf551c178 in _PyEval_EvalFrameDefault (tstate=0x55faf589e3e0 <_PyRuntime+180992>, frame=0x7f5d9062e0d8, throwflag=0)atPython/ceval.c:4206#44 0x000055faf5520c0e in _PyEval_EvalFrame (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>, frame=frame@entry=0x7f5d9062e020,throwflag=throwflag@entry=0)at ./Include/internal/pycore_ceval.h:88#45 0x000055faf5520d26 in _PyEval_Vector (tstate=0x55faf589e3e0 <_PyRuntime+180992>, func=0x7f5d8fe726d0, locals=locals@entry=0x0,args=0x7f5d8fe686a8,argcount=<optimizedout>,kwnames=<optimizedout>)atPython/ceval.c:5826#46 0x000055faf5431288 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>)atObjects/call.c:396#47 0x000055faf5430e33 in _PyVectorcall_Call (tstate=tstate@entry=0x55faf589e3e0 <_PyRuntime+180992>, func=0x55faf5431238 <_PyFunction_Vectorcall>,callable=callable@entry=0x7f5d8fe726d0,tuple=tuple@entry=0x7f5d8fe68690,kwargs=kwargs@entry=0x0)atObjects/call.c:246#48 0x000055faf54311d0 in _PyObject_Call (tstate=0x55faf589e3e0 <_PyRuntime+180992>, callable=callable@entry=0x7f5d8fe726d0,args=args@entry=0x7f5d8fe68690,kwargs=kwargs@entry=0x0)atObjects/call.c:329#49 0x000055faf5431218 in PyObject_Call (callable=callable@entry=0x7f5d8fe726d0, args=args@entry=0x7f5d8fe68690, kwargs=kwargs@entry=0x0)atObjects/call.c:356#50 0x000055faf559cf4d in pymain_run_module (modname=<optimized out>, set_argv0=set_argv0@entry=1) at Modules/main.c:300#51 0x000055faf559da6e in pymain_run_python (exitcode=exitcode@entry=0x7ffe797ef5d4) at Modules/main.c:604#52 0x000055faf559dd56 in Py_RunMain () at Modules/main.c:689#53 0x000055faf559ddcd in pymain_main (args=args@entry=0x7ffe797ef630) at Modules/main.c:719#54 0x000055faf559de93 in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:743#55 0x000055faf53a2762 in main (argc=<optimized out>, argv=<optimized out>) at ./Programs/python.c:15

@chgnrdv
Copy link
ContributorAuthor

chgnrdv commentedNov 3, 2022•
edited
Loading

@ericsnowcurrently
I ran test package in random order on current main multiple times. I also tried to puttest_imp on different positions in list of default and randomly ordered tests. Crash occurs betweentest_imp and the next one iftest_imp is placed on position in test list that is large enough, with the lower bound somewhere between 75-100. Running with-v flag shows (along with python traceback) that the actual test passes, but crash happens during the process of saving some environment shared between tests.
When I run tests with command line arguments similar to which used in CI, i. e.
./python -u -W default -bb -E -E -m test -w -j 1 -u all -W --slowest --fail-env-changed --timeout=1200 -j4 -uall,-cpu
crash doesn't occur.

ericsnowcurrently reacted with thumbs up emoji

@ericsnowcurrently
Copy link
Member

@chgnrdv, sorry for not getting back to you. Is the problem solved by the fix forgh-99578?

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@JelleZijlstraJelleZijlstraJelleZijlstra left review comments

@sweeneydesweeneydesweeneyde left review comments

@ericsnowcurrentlyericsnowcurrentlyericsnowcurrently approved these changes

@brettcannonbrettcannonAwaiting requested review from brettcannon

@encukouencukouAwaiting requested review from encukou

@ncoghlanncoghlanAwaiting requested review from ncoghlan

@warsawwarsawAwaiting requested review from warsaw

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

create_builtin() in _imp module trigger segfault if taking a builtin object as input
6 participants
@chgnrdv@ericsnowcurrently@JelleZijlstra@sweeneyde@bedevere-bot@miss-islington
[8]ページ先頭
©2009-2025 Movatter.jp