Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.7k
Closed
Description
Crash report
What happened?
Take the following simple example C extension:
#definePY_SSIZE_T_CLEAN#include<Python.h>typedefstruct {PyObject_HEAD/* Type-specific fields go here. */}CustomObject;staticPyTypeObjectCustomType= { .ob_base=PyVarObject_HEAD_INIT(NULL,0) .tp_name="weak_bug_repro.Custom", .tp_doc=PyDoc_STR("Custom objects"), .tp_basicsize=sizeof(CustomObject), .tp_itemsize=0, .tp_flags=Py_TPFLAGS_DEFAULT |Py_TPFLAGS_MANAGED_WEAKREF, .tp_new=PyType_GenericNew,};staticintweak_bug_repro_exec(PyObject*m){if (PyType_Ready(&CustomType)<0) {return-1; }if (PyModule_AddObjectRef(m,"Custom", (PyObject*)&CustomType)<0) {return-1; }return0;}staticPyModuleDef_Slotweak_bug_repro_module_slots[]= { {Py_mod_exec,weak_bug_repro_exec}, {0,NULL}};staticPyModuleDefweak_bug_repro_module= { .m_base=PyModuleDef_HEAD_INIT, .m_name="weak_bug_repro", .m_size=0, .m_slots=weak_bug_repro_module_slots,};PyMODINIT_FUNCPyInit_weak_bug_repro(void){returnPyModuleDef_Init(&weak_bug_repro_module);}
After compiling, running the following Python code will crash:
importweak_bug_reproimportweakrefobj=weak_bug_repro.Custom()ref=weakref.ref(obj)
Backtrace for the crash (via gdb on Linux, although it also crashes on macOS, and probably any platform):
Program received signal SIGSEGV, Segmentation fault.0x00007ffff7a0b14b in get_basic_refs (head=0x2000000000000000, refp=refp@entry=0x7fffffffd298, proxyp=proxyp@entry=0x7fffffffd290) at Objects/weakrefobject.c:283283 if (head != NULL && head->wr_callback == NULL) {(gdb) bt#0 0x00007ffff7a0b14b in get_basic_refs (head=0x2000000000000000, refp=refp@entry=0x7fffffffd298, proxyp=proxyp@entry=0x7fffffffd290) at Objects/weakrefobject.c:283#1 0x00007ffff7a0b6b6 in try_reuse_basic_ref (list=<optimized out>, type=type@entry=0x7ffff7de9ae0 <_PyWeakref_RefType>, callback=callback@entry=0x0) at Objects/weakrefobject.c:338#2 0x00007ffff7a0b8d2 in get_or_create_weakref (type=type@entry=0x7ffff7de9ae0 <_PyWeakref_RefType>, obj=0x7ffff6e55650, callback=0x0) at Objects/weakrefobject.c:428#3 0x00007ffff7a0b956 in weakref___new__ (type=0x7ffff7de9ae0 <_PyWeakref_RefType>, args=<optimized out>, kwargs=<optimized out>) at Objects/weakrefobject.c:467#4 0x00007ffff79b529d in type_call (self=0x7ffff7de9ae0 <_PyWeakref_RefType>, args=0x7ffff6e78780, kwds=0x0) at Objects/typeobject.c:2291#5 0x00007ffff7926168 in _PyObject_MakeTpCall (tstate=tstate@entry=0x7ffff7e57200 <_PyRuntime+331232>, callable=callable@entry=0x7ffff7de9ae0 <_PyWeakref_RefType>, args=args@entry=0x7fffffffd608, nargs=<optimized out>, keywords=keywords@entry=0x0) at Objects/call.c:242#6 0x00007ffff792639b in _PyObject_VectorcallTstate (tstate=0x7ffff7e57200 <_PyRuntime+331232>, callable=callable@entry=0x7ffff7de9ae0 <_PyWeakref_RefType>, args=args@entry=0x7fffffffd608, nargsf=<optimized out>, nargsf@entry=9223372036854775809, kwnames=kwnames@entry=0x0) at ./Include/internal/pycore_call.h:167#7 0x00007ffff7926415 in PyObject_Vectorcall (callable=callable@entry=0x7ffff7de9ae0 <_PyWeakref_RefType>, args=args@entry=0x7fffffffd608, nargsf=9223372036854775809, kwnames=kwnames@entry=0x0) at Objects/call.c:327#8 0x00007ffff7a5cd9b in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=0x7ffff7fb7020, throwflag=0) at Python/generated_cases.c.h:1619#9 0x00007ffff7a7b128 in _PyEval_EvalFrame (tstate=tstate@entry=0x7ffff7e57200 <_PyRuntime+331232>, frame=frame@entry=0x7ffff7fb7020, throwflag=throwflag@entry=0) at ./Include/internal/pycore_ceval.h:119#10 0x00007ffff7a7b2e8 in _PyEval_Vector (tstate=tstate@entry=0x7ffff7e57200 <_PyRuntime+331232>, func=func@entry=0x7ffff6e66750, locals=locals@entry=0x7ffff6e74050, args=args@entry=0x0, argcount=argcount@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:1961#11 0x00007ffff7a7b3c5 in PyEval_EvalCode (co=co@entry=0x7ffff6fca260, globals=globals@entry=0x7ffff6e74050, locals=locals@entry=0x7ffff6e74050) at Python/ceval.c:853#12 0x00007ffff7aeda93 in run_eval_code_obj (tstate=tstate@entry=0x7ffff7e57200 <_PyRuntime+331232>, co=co@entry=0x7ffff6fca260, globals=globals@entry=0x7ffff6e74050, locals=locals@entry=0x7ffff6e74050) at Python/pythonrun.c:1365#13 0x00007ffff7aedc47 in run_mod (mod=mod@entry=0x555555680260, filename=filename@entry=0x7ffff6eed9a0, globals=globals@entry=0x7ffff6e74050, locals=locals@entry=0x7ffff6e74050, flags=flags@entry=0x7fffffffd9d8, arena=arena@entry=0x7ffff6ee4040, interactive_src=0x0, generate_new_source=0) at Python/pythonrun.c:1436#14 0x00007ffff7aee4ab in pyrun_file (fp=fp@entry=0x55555555b650, filename=filename@entry=0x7ffff6eed9a0, start=start@entry=257, globals=globals@entry=0x7ffff6e74050, locals=locals@entry=0x7ffff6e74050, closeit=closeit@entry=1, flags=0x7fffffffd9d8) at Python/pythonrun.c:1293#15 0x00007ffff7aefcd6 in _PyRun_SimpleFileObject (fp=fp@entry=0x55555555b650, filename=filename@entry=0x7ffff6eed9a0, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffd9d8) at Python/pythonrun.c:521#16 0x00007ffff7aefec7 in _PyRun_AnyFileObject (fp=fp@entry=0x55555555b650, filename=filename@entry=0x7ffff6eed9a0, closeit=closeit@entry=1, flags=flags@entry=0x7fffffffd9d8) at Python/pythonrun.c:81#17 0x00007ffff7b17aef in pymain_run_file_obj (program_name=program_name@entry=0x7ffff6eeda10, filename=filename@entry=0x7ffff6eed9a0, skip_source_first_line=0) at Modules/main.c:410#18 0x00007ffff7b17bff in pymain_run_file (config=config@entry=0x7ffff7e222b8 <_PyRuntime+114328>) at Modules/main.c:429#19 0x00007ffff7b186cf in pymain_run_python (exitcode=exitcode@entry=0x7fffffffdb3c) at Modules/main.c:694#20 0x00007ffff7b18910 in Py_RunMain () at Modules/main.c:775#21 0x00007ffff7b1896b in pymain_main (args=args@entry=0x7fffffffdb80) at Modules/main.c:805#22 0x00007ffff7b189eb in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:829#23 0x0000555555555142 in main (argc=<optimized out>, argv=<optimized out>) at ./Programs/python.c:15I've initially seen this in Python 3.12 (wherePy_TPFLAGS_MANAGED_WEAKREF was introduced), but I've just tested it against the current main branch, and it still crashes there.
The reason for the crash is the following:
PyType_GenericAlloc(defaulttp_allocof all types) allocates the object by calling_PyType_AllocNoTrack(Objects/typeobject.c)_PyType_AllocNoTrackuses_PyType_PreHeaderSizeto determine how many bytes to allocate in front of thePyObjectstructure (Objects/typeobject.c)_PyType_PreHeaderSizereturns the size of 2 pointers if eitherPy_TPFLAGS_MANAGED_WEAKREForPy_TPFLAGS_MANAGED_DICTis a flagplussizeof(PyGC_Head)if the object supports GC traversal, which is also the size of 2 pointers (technically 2 timesuintptr_) (Include/internal/pycore_object.h)- However,
MANAGED_WEAKREF_OFFSETis hard-defined to be(((Py_ssize_t)sizeof(PyObject *))*-4), which is only correct if thePyGC_Headstruct isalso present, and that is put intotp_weaklistoffsetof the object - And
GET_WEAKREFS_LISTPTRused inObjects/weakrefobject.cfor creating a weak reference then reads from memory that comesbefore the area thatmalloc()allocated from, causing an invalid memory read, causing the crash
In Python 3.12 this would always crash, in 3.13+ this would only crash if GIL is not disabled at compile time, becauseInclude/internal/pycore_typeobject.h definesMANAGED_WEAKREF_OFFSET to be(((Py_ssize_t)sizeof(PyObject *))*-2) ifPy_GIL_DISABLED is defined.
The correct behavior should be be to set thetp_weaklistoffset correctly according to whether_PyType_IS_GC is true or not - but since there are several explicit checks fortp_weaklistoffset against the constantMANAGED_WEAKREF_OFFSET, this will probably require some level of refactoring.
CPython versions tested on:
3.12
Operating systems tested on:
No response
Output from running 'python -VV' on the command line:
No response