Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.7k
Open
Description
Thecopy.copy andcopy.deepcopy operations running on different objects in different threads should be able to run independently. The scaling is not good: output of theftscalingbench.py with added benchmarks:
Running benchmarks with 8 threadsshallow_copy 4.6x slowerdeepcopy 2.3x slowerobject_cfunction 5.9x fastercmodule_function 5.9x fasterobject_lookup_special 5.5x fastermult_constant 5.6x fastergenerator 4.9x fasterpymethod 5.2x fasterpyfunction 4.5x fastermodule_function 5.0x fasterload_string_const 5.6x fasterload_tuple_const 5.9x fastercreate_pyobject 2.3x fastercreate_closure 7.1x fastercreate_dict 3.3x fasterthread_local_read 3.3x fasterThere are at least two reasons:
- The
copymodule uses module level variables in thecopy.copyandcopy.deepcopymethods. - The
_copy_atomic_types(and some similar data structures) is asetwhich requires locking for membership testing.
Linked PRs
- gh-132657: Add free-threading scaling benchmark for copy.copy and copy.deepcopy #132658
- gh-132657: Avoid locking in frozenset.__contains__ #132659
- gh-132657: Avoid locks and refcounts in frozenset operations #136107
- GH-132657: Add lock-free set contains implementation #132290
- gh-132657: Make deepcopy and copy scale with free-threading #138429
- gh-132657: optimize
PySet_Containsforfrozensets#141183 - [3.14] gh-132657: avoid locks and refcounting in
frozensetlookups (GH-136107) #141772 - [3.14] gh-132657: optimize
PySet_Containsforfrozenset(GH-141183) #141773 - gh-132657: revert lock-free set contains #142733
- gh-132657: Use stronger memory ordering for so->mask. #142735
- gh-132657: Add maybe_enable_deferred_ref_count(). #142843