Bug report

Bug description:

Reported by@pablogsal /@godlygeek from memray

Stack trace:
https://gist.github.com/pablogsal/513fa8b0c29cda852ce11c86ce3b1345

We have two threads, the main thread (M) and a daemon thread (D). The main thread starts_Py_Finalize() and performs a global stop the world. The daemon thread is disabling profiling and so tries to performa a stop-the-world specific to it's interpreter:

M:_PyEval_StopTheWorldAll():
M: acquiresruntime->stoptheworld->mutex
M: acquires RW lockruntime->stoptheworld_mutex in W (exclusive) mode
M: ... waits on threads
D:_PyEval_StopTheWorld(interp):
D: acquiresinterp->stoptheworld->mutex
D: ... blocks trying to acquireruntime->stoptheworld_mutex in R mode. Later, the daemon thread will hang in_PyThreadState_HangThread() when trying to re-attach it's thread state.
M:_PyEval_StopTheWorldAll() finishes, marks the interpreter as finalizing
M: ...
M: calls_PyGC_CollectNoFail() which tries to run_PyEval_StopTheWorld(interp)
M: ... blocks trying to acquireinterp->stoptheworld->mutex, which is still held by the daemon thread!

Deadlock! Summary:

The daemon thread holdsinterp->stoptheworld->mutex and is hanging because the interpreter is shutting down.

The main thread is trying to perform the shutdown procedure, including calling the GC a few times, which requiresinterp->stoptheworld->mutex.

Fix???

• Release the previously acquiredinterp->stoptheworld->mutex when hanging the thread if necessary? Crosses a bunch of abstraction barriers, which is messy and tricky

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-137433: Fix deadlock with stop-the-world and daemon threads #137735
• [3.14] gh-137433: Fix deadlock with stop-the-world and daemon threads (gh-137735) #138965