Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32k
[3.13] GH-132380: Avoid locking in _PyType_LookupRef()#132381
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Change find_name_in_mro() so we don't need hold TYPE_LOCK. Change_PyType_LookupRef() so that TYPE_LOCK is acquired only if the typedoesn't have a version tag already assigned. If the version is alreadyassigned, we can avoid acquiring TYPE_LOCK.
I'm not sure I fully understand this, but a few preliminary thoughts:
|
Yes, I think the replacement could happen. If I understand the interleaving you are thinking of, two lookups on the same type could be racing with each other, call them A and B. The A starts first and assigns a version tag that will later be out-of-date. The B lookup starts after but completes first, adding its valid entry to the cache. The A lookup completes after, overriding the good cache entry with a stale one. I don't see this being a major problem. We validate the cache entry on lookup, comparing entry version and type version, and so it's not correctness issue. We would just have to do the slow lookup again and re-add it to the cache. Maybe I don't understand the interleaving you are thinking of. I don't think it's a permanent replacement. Another scenario is that you replace a valid cache entry with a stale one due to hash table collision. Again, I don't think that's permanent. The next lookup for that previous entry will miss the cache, do the slow lookup and re-add the entry to the cache. We could avoid adding the invalid cache entry by checking if
The main branch has the same contention problem. I have a similar fix as part ofGH-131174. I can make a separate PR if we want to fix it separately. I'm hoping I'm close to makingGH-131174 acceptable to merge.
If my approach is safe, I think it provides benefits beyond just the non-interned string case. I would expect that most types don't get mutated after a certain point. With this change, once their type version tag is assigned, |
Thinking more about this change: because the types lock is not held during |
Closing, this need a re-think for 3.15. With this PR as is, it introduces refcount contention on tp_mro . The provisional plan is to handle the non-interned string lookup by caching them. |
Uh oh!
There was an error while loading.Please reload this page.
Change
find_name_in_mro()
so we don't need holdTYPE_LOCK
. Change_PyType_LookupRef()
so thatTYPE_LOCK
is acquired only if the type doesn't have a version tag already assigned. If the version is already assigned, we can avoid acquiringTYPE_LOCK
._PyType_LookupRef
#132380