Memory usage numbers (proposed fix explained below):

             VmHWMGIL      135104 kB  - normal GIL-enabled baselinenoGIL   6702788 kB  - free-threaded current QSBR behaviorfix      517760 kB  - free-threaded with _PyMem_ProcessDelayed() in _Py_HandlePending()

Test script:

importthreadingfromqueueimportQueuedefthrdfunc(queue):whileTrue:l=queue.get()l.append(0)# force resize in non-parent thread which will free using _PyMem_FreeDelayed()queue=Queue(maxsize=2)threading.Thread(target=thrdfunc,args=(queue,)).start()whileTrue:l= [None]*int(3840*2160*3/8)# sys.getsizeof(l) ~= 3840*2160*3 bytesqueue.put(l)

Delayed memory free checks (and subsequent frees if applicable) currently only occur in one of two situations:

• Garbage collection, which doesn't trigger often enough in this script, though manual trigger solves problem.
• On a_PyMem_FreeDelayed() when the number of pending delayed free memory blocks reaches exactly 254. And then it waits another 254 frees even if could not free any pending blocks this time, which is a lot for big buffers.

This works great for many small objects, but with larger buffers these can accumulate quickly, so more frequent checks should be done.

I tried a few things but_PyMem_ProcessDelayed() added to_Py_HandlePending() seems to work well and be safe and aQSBR_QUIESCENT_STATE has just been reported so there is a fresh chance to actually free. Seems to happen often enough that memory usage is kept down, and if nothing to free then_PyMem_ProcessDelayed() is super-cheap.

Another option would be to track the amount of pending memory to be freed and increase the frequency of free attempts if that number gets too large, but to start with this small change seems to solved the problem well enough. Could also schedule GC if pending frees get too high, but that seems like a roundabout way to arrive at_PyMem_ProcessDelayedNoDealloc().

Performance as checked bypyperformance full suite is unchanged with the fix (literally 0.17% better avg, so noise).

• Issue:Excessive QSBR memory usage when delay freeing large blocks. #132519