Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.3k
Description
For now, I can only occationally observe the segfault on github actions. This is an issue that's not easy to reproduce, but I tried to understand the cause of it.
The direct cause would be indeduce_unreachable ingcmodule.c. In that function,gc tries to find cycles by traversing objects, including frame, which uses_PyFrame_Traverse for all its objects. In_PyFrame_Traverse, it usesframe->stacktop as the index range for all the locals and temporary data on stack(not sure if that's on purpose). However,frame->stacktop is not updated in real-time, which means the object it traverses might not be valid.
For example, inFOR_ITER dispatch code, there's aPy_DECREF(iter); STACK_SHRINK(1); when the iterator is exhausted. However,STACK_SHIRNK only increasesstack_pointer, notframe->stacktop. At this point, theiter that's just freed will be traversed during garbage collection.
There might be something I missed because it's not trivial to reproduce this, but I got a demo that could reproduce this issue occasionally.
frommultiprocessingimportPoolimportsysdeftracefunc(frame,*args):a=100**100defpool_worker(item):return {"a":1}defpool_indexer(path):item_count=0withPool(processes=8)aspool:foriinpool.imap(pool_worker,range(1,2000),chunksize=10):item_count=item_count+1sys.setprofile(tracefunc)pool_indexer(10)
It might have something to do with the profile function, I think I can only reproduce this with it. You need to enable--with-address-sanitizer to find an error ofERROR: AddressSanitizer: heap-use-after-free on address. Normally inPy_TYPE Include/object.h:135, where the code dereferencedob, which could be freed already.
The memory it points to is often benign so I'm not able to reliably generate SegFaults, but in theory, this is a memory violation.
Python Version: cpython/main
OS Version: Ubuntu 20 on WSL