Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.7k
gh-119127: functools.partial placeholders#119827
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Changes from70 commits
ee7333c8bcc462c67c9b4680d9009591ff5067e9388af20b3607a0b1f55801e58941453722e07a79c2af12aaa7292c767b496a9d238d9c11707b95714b38ca32bca198576493a3fd2d608529936fea348caec6e8115b8c53f5f00b202c9292c16d38400ff558ccc38fe7c82c7c9b7ef3e59d7117bfc5917957a978aaee6afe8e0ad00dd80ed352cfa9038ed549b8c71bc1fdbd30672211185510266b4fadd58a125971fbb9033650d31e5d1a3d39b09e4c5df16f12f882dd600f9cb653d255524404044e800217b38ee45011f47db3c872bdfd16189a6c6ef21c8d73ea8bd3ae70e47ed2eacf5ef78d8d30a8640e6e3d28266c305d14bf68cee642d58d6c28e8744bcbb8964704881ae6c3ad7d95e5d484File filter
Filter by extension
Conversations
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -348,7 +348,7 @@ The :mod:`functools` module defines the following functions: | ||
| The :func:`partial` is used for partial function application which "freezes" | ||
| some portion of a function's arguments and/or keywords resulting in a new object | ||
| with a simplified signature. For example, :func:`partial` can be used to create | ||
dg-pb marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| a callable that behaves like the :func:`int` function where the *base* argument | ||
| defaults to two: | ||
| @@ -358,6 +358,49 @@ The :mod:`functools` module defines the following functions: | ||
| >>> basetwo('10010') | ||
| 18 | ||
| If :data:`Placeholder` sentinels are present in *args*, they will be filled first | ||
| when :func:`partial` is called. This allows custom selection of positional arguments | ||
| to be pre-filled when constructing a :ref:`partial object <partial-objects>`. | ||
| If :data:`!Placeholder` sentinels are present, all of them must be filled at call time: | ||
| >>> from functools import partial, Placeholder | ||
dg-pb marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page.
picnixz marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| >>> say_to_world = partial(print, Placeholder, Placeholder, "world!") | ||
| >>> say_to_world('Hello', 'dear') | ||
| Hello dear world! | ||
| Calling ``say_to_world('Hello')`` would raise a :exc:`TypeError`, because | ||
| only one positional argument is provided, while there are two placeholders | ||
| in :ref:`partial object <partial-objects>`. | ||
| Successive :func:`partial` applications fill :data:`!Placeholder` sentinels | ||
| of the input :func:`partial` objects with new positional arguments. | ||
| A place for positional argument can be retained by inserting new | ||
| :data:`!Placeholder` sentinel to the place held by previous :data:`!Placeholder`: | ||
| >>> from functools import partial, Placeholder as _ | ||
| >>> remove = partial(str.replace, _, _, '') | ||
| >>> message = 'Hello, dear dear world!' | ||
| >>> remove(message, ' dear') | ||
| 'Hello, world!' | ||
| >>> remove_dear = partial(remove, _, ' dear') | ||
| >>> remove_dear(message) | ||
| 'Hello, world!' | ||
| >>> remove_first_dear = partial(remove_dear, _, 1) | ||
| >>> remove_first_dear(message) | ||
| 'Hello, dear world!' | ||
| Note, :data:`!Placeholder` has no special treatment when used for keyword | ||
| argument of :data:`!Placeholder`. | ||
| .. versionchanged:: 3.14 | ||
| Added support for :data:`Placeholder` in positional arguments. | ||
| .. data:: Placeholder | ||
| A singleton object used as a sentinel to reserve a place | ||
| for positional arguments when calling :func:`partial` | ||
| and :func:`partialmethod`. | ||
dg-pb marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| .. class:: partialmethod(func, /, *args, **keywords) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -17,6 +17,7 @@ | ||
| from abc import get_cache_token | ||
| from collections import namedtuple | ||
| # import types, weakref # Deferred to single_dispatch() | ||
| from operator import itemgetter | ||
dg-pb marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| from reprlib import recursive_repr | ||
| from types import MethodType | ||
| from _thread import RLock | ||
| @@ -274,43 +275,125 @@ def reduce(function, sequence, initial=_initial_missing): | ||
| ### partial() argument application | ||
| ################################################################################ | ||
| class _PlaceholderType: | ||
| """The type of the Placeholder singleton. | ||
| Used as a placeholder for partial arguments. | ||
| """ | ||
| __instance = None | ||
dg-pb marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| __slots__ = () | ||
| def __init_subclass__(cls, *args, **kwargs): | ||
| raise TypeError(f"type '{cls.__name__}' is not an acceptable base type") | ||
| def __new__(cls): | ||
| if cls.__instance is None: | ||
| cls.__instance = object.__new__(cls) | ||
| return cls.__instance | ||
Comment on lines +290 to +293 Member There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. I think this is overcomplication. The user has no reasons to create an instance of private class If you want to add some guards here, just make ContributorAuthor There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. This was to mimic C implementation, which has the behaviour of I think the question is: "Is it a good practice for a non-trivial sentinel to be singleton, i.e. If yes and this sentinel is considered non-trivial, then this is as good as it can get for now and protection issues can be sorted out together with further developments in this area. If no, then this needs to be changed for both C and Python. @rhettinger has suggested this initially and I like this behaviour (and adapted it to my own sentinels). It would be good if you together could come to agreement before I make any further changes here. Member There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. This is not required for its main function, and this complicates both implementations. It is better to implement only necessary parts. If later we will find a need of this feature, it will be easier to add it than to remove it. Strictly speaking, making the Placeholder class non-inheritable and non-instantiable is not required. But it is easy to implement. I hope Raymond will change his opinion on this. ContributorAuthor There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. What you are saying makes sense, but at the same time I like current behaviour and if sentinels were standardised and their creation was made more convenient I think this elegant behaviour would be nice to get by default. I am neutral by now on this specific case. Well, slightly negative just because I put thought and effort into this and simply like it. | ||
| def __repr__(self): | ||
| return 'Placeholder' | ||
| def __reduce__(self): | ||
| return 'Placeholder' | ||
| Placeholder = _PlaceholderType() | ||
| def _partial_prepare_merger(args): | ||
| if not args: | ||
| return 0, None | ||
| nargs = len(args) | ||
| order = [] | ||
| j = nargs | ||
| for i, a in enumerate(args): | ||
| if a is Placeholder: | ||
| order.append(j) | ||
| j += 1 | ||
| else: | ||
| order.append(i) | ||
| phcount = j - nargs | ||
| merger = itemgetter(*order) if phcount else None | ||
| return phcount, merger | ||
| def _partial_new(cls, func, /, *args, **keywords): | ||
| if issubclass(cls, partial): | ||
| base_cls = partial | ||
| if not callable(func): | ||
| raise TypeError("the first argument must be callable") | ||
| else: | ||
| base_cls = partialmethod | ||
| # func could be a descriptor like classmethod which isn't callable | ||
| if not callable(func) and not hasattr(func, "__get__"): | ||
| raise TypeError(f"the first argument {func!r} must be a callable " | ||
| "or a descriptor") | ||
| if args and args[-1] is Placeholder: | ||
| raise TypeError("trailing Placeholders are not allowed") | ||
dg-pb marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| if isinstance(func, base_cls): | ||
| pto_phcount = func._phcount | ||
| tot_args = func.args | ||
| if args: | ||
| tot_args += args | ||
| if pto_phcount: | ||
| # merge args with args of `func` which is `partial` | ||
| nargs = len(args) | ||
| if nargs < pto_phcount: | ||
| tot_args += (Placeholder,) * (pto_phcount - nargs) | ||
| tot_args = func._merger(tot_args) | ||
| if nargs > pto_phcount: | ||
| tot_args += args[pto_phcount:] | ||
| phcount, merger = _partial_prepare_merger(tot_args) | ||
| else: # works for both pto_phcount == 0 and != 0 | ||
| phcount, merger = pto_phcount, func._merger | ||
| keywords = {**func.keywords, **keywords} | ||
| func = func.func | ||
| else: | ||
| tot_args = args | ||
| phcount, merger = _partial_prepare_merger(tot_args) | ||
| self = object.__new__(cls) | ||
| self.func = func | ||
| self.args = tot_args | ||
| self.keywords = keywords | ||
| self._phcount = phcount | ||
| self._merger = merger | ||
| return self | ||
| def _partial_repr(self): | ||
| cls = type(self) | ||
| module = cls.__module__ | ||
| qualname = cls.__qualname__ | ||
| args = [repr(self.func)] | ||
| args.extend(map(repr, self.args)) | ||
| args.extend(f"{k}={v!r}" for k, v in self.keywords.items()) | ||
| return f"{module}.{qualname}({', '.join(args)})" | ||
| # Purely functional, no descriptor behaviour | ||
| class partial: | ||
| """New function with partial application of the given arguments | ||
| and keywords. | ||
| """ | ||
| __slots__ = ("func", "args", "keywords", "_phcount", "_merger", | ||
| "__dict__", "__weakref__") | ||
| __new__ = _partial_new | ||
| __repr__ = recursive_repr()(_partial_repr) | ||
| def __call__(self, /, *args, **keywords): | ||
| phcount = self._phcount | ||
| if phcount: | ||
| try: | ||
| pto_args = self._merger(self.args + args) | ||
| args = args[phcount:] | ||
| except IndexError: | ||
| raise TypeError("missing positional arguments " | ||
| "in 'partial' call; expected " | ||
| f"at least {phcount}, got {len(args)}") | ||
| else: | ||
| pto_args = self.args | ||
| keywords = {**self.keywords, **keywords} | ||
| return self.func(*pto_args, *args, **keywords) | ||
| def __get__(self, obj, objtype=None): | ||
| if obj is None: | ||
| @@ -332,6 +415,10 @@ def __setstate__(self, state): | ||
| (namespace is not None and not isinstance(namespace, dict))): | ||
| raise TypeError("invalid partial state") | ||
| if args and args[-1] is Placeholder: | ||
| raise TypeError("trailing Placeholders are not allowed") | ||
| phcount, merger = _partial_prepare_merger(args) | ||
| args = tuple(args) # just in case it's a subclass | ||
| if kwds is None: | ||
| kwds = {} | ||
| @@ -344,53 +431,40 @@ def __setstate__(self, state): | ||
| self.func = func | ||
| self.args = args | ||
| self.keywords = kwds | ||
| self._phcount = phcount | ||
| self._merger = merger | ||
| try: | ||
| from _functools import partial, Placeholder, _PlaceholderType | ||
| except ImportError: | ||
| pass | ||
| # Descriptor version | ||
| class partialmethod: | ||
| """Method descriptor with partial application of the given arguments | ||
| and keywords. | ||
| Supports wrapping existing descriptors and handles non-descriptor | ||
| callables as instance methods. | ||
| """ | ||
| __new__ = _partial_new | ||
| __repr__ = _partial_repr | ||
| def _make_unbound_method(self): | ||
| def _method(cls_or_self, /, *args, **keywords): | ||
| phcount = self._phcount | ||
| if phcount: | ||
| try: | ||
| pto_args = self._merger(self.args + args) | ||
| args = args[phcount:] | ||
| except IndexError: | ||
| raise TypeError("missing positional arguments " | ||
| "in 'partialmethod' call; expected " | ||
| f"at least {phcount}, got {len(args)}") | ||
| else: | ||
| pto_args = self.args | ||
| keywords = {**self.keywords, **keywords} | ||
| return self.func(cls_or_self, *pto_args, *args, **keywords) | ||
| _method.__isabstractmethod__ = self.__isabstractmethod__ | ||
| _method.__partialmethod__ = self | ||
| return _method | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1932,7 +1932,12 @@ def _signature_get_partial(wrapped_sig, partial, extra_args=()): | ||
| if param.kind is _POSITIONAL_ONLY: | ||
| # If positional-only parameter is bound by partial, | ||
| # it effectively disappears from the signature | ||
| # However, if it is a Placeholder it is not removed | ||
| # And also looses default value | ||
| if arg_value is functools.Placeholder: | ||
| new_params[param_name] = param.replace(default=_empty) | ||
| else: | ||
| new_params.pop(param_name) | ||
| continue | ||
| if param.kind is _POSITIONAL_OR_KEYWORD: | ||
| @@ -1954,7 +1959,17 @@ def _signature_get_partial(wrapped_sig, partial, extra_args=()): | ||
| new_params[param_name] = param.replace(default=arg_value) | ||
| else: | ||
| # was passed as a positional argument | ||
| # Do not pop if it is a Placeholder | ||
| # also change kind to positional only | ||
| # and remove default | ||
| if arg_value is functools.Placeholder: | ||
dg-pb marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| new_param = param.replace( | ||
| kind=_POSITIONAL_ONLY, | ||
| default=_empty | ||
| ) | ||
| new_params[param_name] = new_param | ||
| else: | ||
| new_params.pop(param_name) | ||
| continue | ||
| if param.kind is _KEYWORD_ONLY: | ||
| @@ -2448,6 +2463,11 @@ def _signature_from_callable(obj, *, | ||
| sig_params = tuple(sig.parameters.values()) | ||
| assert (not sig_params or | ||
| first_wrapped_param is not sig_params[0]) | ||
| # If there were placeholders set, | ||
| # first param is transformed to positional only | ||
| if partialmethod.args.count(functools.Placeholder): | ||
| first_wrapped_param = first_wrapped_param.replace( | ||
| kind=Parameter.POSITIONAL_ONLY) | ||
| new_params = (first_wrapped_param,) + sig_params | ||
| return sig.replace(parameters=new_params) | ||
Uh oh!
There was an error while loading.Please reload this page.