Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.3k
gh-85160: Reduce memory usage ofsingledispatchmethod when owner instances cannot be weakref'd#107706
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
…stances cannot be weakref'd
singledispatchmethod when instances cannot be weakref'dsingledispatchmethod when owner instances cannot be weakref'dAlternatively, you can get rid of diff --git a/Lib/functools.py b/Lib/functools.pyindex 2a8a69b3c5..5622eb6467 100644--- a/Lib/functools.py+++ b/Lib/functools.py@@ -935,7 +935,6 @@ def __init__(self, func): self.dispatcher = singledispatch(func) self.func = func self._method_cache = weakref.WeakKeyDictionary()- self._all_weakrefable_instances = True def register(self, cls, method=None): """generic_method.register(cls, func) -> func@@ -945,11 +944,11 @@ def register(self, cls, method=None): return self.dispatcher.register(cls, func=method) def __get__(self, obj, cls=None):- if self._all_weakrefable_instances:+ if self._method_cache: try: _method = self._method_cache[obj] except TypeError:- self._all_weakrefable_instances = False+ self._method_cache = None except KeyError: pass else:@@ -963,7 +962,7 @@ def _method(*args, **kwargs): _method.register = self.register update_wrapper(_method, self.func)- if self._all_weakrefable_instances:+ if self._method_cache is not None: self._method_cache[obj] = _method return _method It reduces memory even more, and in all cases. |
Very nice. And the performance seems to be the same. The first line of def__get__(self,obj,cls=None):ifself._method_cacheisnotNone: |
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
951a87e to3d8d4baCompareOr you need to catch TypeError when save in the cache: ifself._method_cacheisnotNone:try:self._method_cache[obj]=_methodexceptTypeError:self._method_cache=None |
With this diff applied to my PR branch: Diff--- a/Lib/functools.py+++ b/Lib/functools.py@@ -945,7 +945,7 @@ def register(self, cls, method=None): return self.dispatcher.register(cls, func=method) def __get__(self, obj, cls=None):- if self._method_cache is not None:+ if self._method_cache: try: _method = self._method_cache[obj] except TypeError:@@ -963,8 +963,10 @@ def _method(*args, **kwargs): _method.register = self.register update_wrapper(_method, self.func)- if self._method_cache is not None:+ try: self._method_cache[obj] = _method+ except TypeError:+ pass return _method Benchmark scriptimportpyperfsetup="""from functools import singledispatch, singledispatchmethodclass Test: @singledispatchmethod def go(self, item, arg): pass @go.register def _(self, item: int, arg): return item + argclass Slot: __slots__ = ('a', 'b') @singledispatchmethod def go(self, item, arg): pass @go.register def _(self, item: int, arg): return item + argt = Test()s= Slot()"""runner=pyperf.Runner()runner.timeit(name="bench singledispatchmethod",stmt="""_ = t.go(1, 1)""",setup=setup )runner.timeit(name="bench singledispatchmethod slots",stmt="""_ = s.go(1, 1)""",setup=setup ) |
Simpler |
I mean that you keep |
I still measure about a 14% overhead from this approach compared to my current PR. (Tested on a PGO-optimised non-debug build on Windows, with a quiet machine.) I'm inclined to stick with how it is currently -- it also feels like slightly cleaner code if we have only one |
With the test for Anyway, your current code is better. I just explained why I intentionally did not use |
Yep. I'm also surprised there was that difference. |
Thanks for the review, Serhiy! |
Uh oh!
There was an error while loading.Please reload this page.
No news, since this is just a small followup to#107148, which is a new feature in Python 3.13
See#107148 (comment) (cc.@eendebakpt)