Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32k
Description
Feature or enhancement
Proposal:
As part of adding free-threading support to LibCST, we noticed there is a lot of lock contention onTYPE_LOCK
inside the_PyType_LookupRef
function. In the LibCST, the common "visitor" pattern is used. For example, in the_visitors.py
module there is the code:
visit_func = getattr(self, f"visit_{type(node).__name__}", None)
The second argument togetattr()
is a non-interned string and it causes the cached and lock-free path of_PyType_LookupRef()
never to be taken. Instead, theTYPE_LOCK
mutex is acquired on each lookup. This obviously scales very badly if there are multiple threads looking up class methods using this pattern.
Testing was done with Python 3.13 but I believe the same issue exists with 3.14.
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Linked PRs
- [3.13] GH-132380: Avoid locking in _PyType_LookupRef() #132381
- gh-127266: avoid data races when updating type slots #131174
- [3.13] GH-132380: Add optimization for non-interned type lookup. #132651
- GH-132380: Add optimization for non-interned type lookup. #132652
- gh-132380: Use unicode hash/compare for the mcache. #133669