Movatterモバイル変換


[0]ホーム

URL:


Following system colour schemeSelected dark colour schemeSelected light colour scheme

Python Enhancement Proposals

PEP 468 – Preserving the order of **kwargs in a function.

Author:
Eric Snow <ericsnowcurrently at gmail.com>
Discussions-To:
Python-Ideas list
Status:
Final
Type:
Standards Track
Created:
05-Apr-2014
Python-Version:
3.6
Post-History:
05-Apr-2014, 08-Sep-2016
Resolution:
Python-Dev message

Table of Contents

Abstract

The **kwargs syntax in a function definition indicates that theinterpreter should collect all keyword arguments that do not correspondto other named parameters. However, Python does not preserved theorder in which those collected keyword arguments were passed to thefunction. In some contexts the order matters. This PEP dictates thatthe collected keyword arguments be exposed in the function body as anordered mapping.

Motivation

Python’s **kwargs syntax in function definitions provides a powerfulmeans of dynamically handling keyword arguments. In some applicationsof the syntax (seeUse Cases), the semantics applied to thecollected keyword arguments requires that order be preserved.Unsurprisingly, this is similar to how OrderedDict is related to dict.

Currently to preserved the order you have to do so manually andseparately from the actual function call. This involves building anordered mapping, whether an OrderedDict or an iterable of 2-tuples,which is then passed as a single argument to the function.[1]

With the capability described in this PEP, that boilerplate would nolonger be required.

For comparison, currently:

kwargs=OrderedDict()kwargs['eggs']=......defspam(a,kwargs):...

and with this proposal:

defspam(a,**kwargs):...

Alyssa (Nick) Coghlan, speaking of some of the uses cases, summed it up well[2]:

These*can*allbedonetoday,but*not*byusingkeywordarguments.Inmyview,theproblemtobeaddressedisthatkeywordarguments*look*liketheyshouldworkforthesecases,becausetheyhaveadefiniteorderinthesourcecode.Theonlyreasontheydon't workisbecausetheinterpreterthrowsthatorderinginformationaway.It's a textbook case of a language feature becoming an attractivenuisanceinsomecircumstances:thesimpleandobvioussolutionfortheaboveusecases*doesn't actually work* for reasons that aren'tobviouslyclearifyoudon't have a firm grasp of Python'sadmittedlycomplicatedargumenthandling.

This observation is supported by the appearance of this proposal overthe years and the numerous times that people have been confused by theconstructor for OrderedDict.[3][4][5]

Use Cases

As Alyssa noted, the current behavior of **kwargs is unintuitive incases where one would expect order to matter. Aside from more specificcases outlined below, in general “anything else where you want tocontrol the iteration orderand set field names and values in a singlecall will potentially benefit.”[6] That matters in thecase of factories (e.g. __init__()) for ordered types.

Serialization

Obviously OrderedDict would benefit (both __init__() and update()) fromordered kwargs. However, the benefit also extends to serializationAPIs[2]:

Inthecontextofserialisation,onekeylessonwehavelearnedisthatarbitraryorderingisaproblemwhenyouwanttominimisespuriousdiffs,andsortingisn't a simple solution.Toolslikedoctestdon't tolerate spurious diffs at all, but areoftenamenabletoasortingbasedanswer.Thecaseswhereitwouldbehighlydesirabletobeableusekeywordargumentstocontroltheorderofdisplayofacollectionofkeyvaluepairsareoneslike:*printingoutkey:valuepairsinCLIoutput*mappingsemanticnamestocolumnorderinaCSV*serialisingattributesandelementsinparticularordersinXML*serialisingmapkeysinparticularordersinhumanreadableformatslikeJSONandYAML(particularlywhenthey're going to be placedundersourcecontrol)

Debugging

In the words of Raymond Hettinger[7]:

Itmakesiteasiertodebugiftheargumentsshow-upintheordertheywerecreated.AFAICT,nopurposeisservedbyscramblingthem.

Other Use Cases

  • Mock objects.[8]
  • Controlling object presentation.
  • Alternate namedtuple() where defaults can be specified.
  • Specifying argument priority by order.

Concerns

Performance

As already noted, the idea of ordered keyword arguments has come up ona number of occasions. Each time it has been met with the sameresponse, namely that preserving keyword arg order would have asufficiently adverse effect on function call performance that it’s notworth doing. However, Guido noted the following[9]:

Making**kwdsorderedisstillopen,butrequirescarefuldesignandimplementationtoavoidslowingdownfunctioncallsthatdon't benefit.

As will be noted below, there are ways to work around this at theexpense of increased complication. Ultimately the simplest approach isthe one that makes the most sense: pack collected key word argumentsinto an OrderedDict. However, without a C implementation of OrderedDictthere isn’t much to discuss. That changed in Python 3.5.[10]

Note: in Python 3.6 dict is order-preserving. This virtually eliminatesperformance concerns.

Other Python Implementations

Another important issue to consider is that new features must becognizant of the multiple Python implementations. At some point each ofthem would be expected to have implemented ordered kwargs. In thisregard there doesn’t seem to be an issue with the idea.[11]An informal survey of the major Python implementations has indicatedthat this feature will not be a significant burden.

Specification

Starting in version 3.6 Python will preserve the order of keywordarguments as passed to a function. To accomplish this the collectedkwargs will now be an ordered mapping. Note that this does not necessarilymean OrderedDict. dict in CPython 3.6 is now ordered, similar to PyPy.

This will apply only to functions for which the definition uses the**kwargs syntax for collecting otherwise unspecified keywordarguments. Only the order of those keyword arguments will bepreserved.

Relationship to **-unpacking syntax

The ** unpacking syntax in function calls has no special connection withthis proposal. Keyword arguments provided by unpacking will be treatedin exactly the same way as they are now: ones that match definedparameters are gather there and the remainder will be collected into theordered kwargs (just like any other unmatched keyword argument).

Note that unpacking a mapping with undefined order, such as dict, willpreserve its iteration order like normal. It’s just that the order willremain undefined. The ordered mapping into which the unpacked key-valuepairs will then be packed will not be able to provide any alternateordering. This should not be surprising.

There have been brief discussions of simply passing these mappingsthrough to the functions kwargs without unpacking and repacking them,but that is both outside the scope of this proposal and probably a badidea regardless. (There is a reason those discussions were brief.)

Relationship to inspect.Signature

Signature objects should need no changes. Thekwargs parameter ofinspect.BoundArguments (returned by Signature.bind() andSignature.bind_partial()) will change from a dict to an OrderedDict.

C-API

No changes.

Syntax

No syntax is added or changed by this proposal.

Backward-Compatibility

The following will change:

  • iteration order of kwargs will now be consistent (except of course inthe case described above)

Reference Implementation

For CPython there’s nothing to do.

Alternate Approaches

Opt-out Decorator

This is identical to the current proposal with the exception that Pythonwould also provide a decorator in functools that would cause collectedkeyword arguments to be packed into a normal dict instead of anOrderedDict.

Prognosis:

This would only be necessary if performance is determined to besignificantly different in some uncommon cases or that there are otherbackward-compatibility concerns that cannot be resolved otherwise.

Opt-in Decorator

The status quo would be unchanged. Instead Python would provide adecorator in functools that would register or mark the decoratedfunction as one that should get ordered keyword arguments. Theperformance overhead to check the function at call time would bemarginal.

Prognosis:

The only real down-side is in the case of function wrappers factories(e.g. functools.partial and many decorators) that aim to perfectlypreserve keyword arguments by using kwargs in the wrapper definitionand kwargs unpacking in the call to the wrapped function. Each wrapperwould have to be updated separately, though having functools.wraps() dothis automaticallywould help.

__kworder__

The order of keyword arguments would be stored separately in a list atcall time. The list would be bound to __kworder__ in the functionlocals.

Prognosis:

This likewise complicates the wrapper case.

Compact dict with faster iteration

Raymond Hettinger has introduced the idea of a dict implementation thatwould result in preserving insertion order on dicts (until the firstdeletion). This would be a perfect fit for kwargs.[5]

Prognosis:

The idea is still uncertain in both viability and timeframe.

Note that Python 3.6 now has this dict implementation.

***kwargs

This would add a new form to a function’s signature as a mutuallyexclusive parallel to **kwargs. The new syntax, ***kwargs (notethat there are three asterisks), would indicate that kwargs shouldpreserve the order of keyword arguments.

Prognosis:

New syntax is only added to Python under the mostdire circumstances.With other available solutions, new syntax is not justifiable.Furthermore, like all opt-in solutions, the new syntax would complicatethe pass-through case.

annotations

This is a variation on the decorator approach. Instead of using adecorator to mark the function, you would use a function annotation on**kwargs.

Prognosis:

In addition to the pass-through complication, annotations have beenactively discouraged in Python core development. Use of annotations toopt-in to order preservation runs the risk of interfering with otherapplication-level use of annotations.

dict.__order__

dict objects would have a new attribute,__order__ that would defaultto None and that in the kwargs case the interpreter would use in thesame way as described above for __kworder__.

Prognosis:

It would mean zero impact on kwargs performance but the change would bepretty intrusive (Python uses dict a lot). Also, for the wrapper casethe interpreter would have to be careful to preserve__order__.

KWArgsDict.__order__

This is the same as thedict.__order__ idea, but kwargs would be aninstance of a new minimal dict subclass that provides the__order__attribute. dict would instead be unchanged.

Prognosis:

Simply switching to OrderedDict is a less complicated and more intuitivechange.

Acknowledgements

Thanks to Andrew Barnert for helpful feedback and to the participants ofall the past email threads.

Footnotes

[1]
Alternately, you could also replace ** in your function definitionwith * and then pass in key/value 2-tuples. This has the advantageof not requiring the keys to be valid identifier strings. Seehttps://mail.python.org/pipermail/python-ideas/2014-April/027491.html.

References

[2] (1,2)
https://mail.python.org/pipermail/python-ideas/2014-April/027512.html
[3]
https://mail.python.org/pipermail/python-ideas/2009-April/004163.html

https://mail.python.org/pipermail/python-ideas/2010-October/008445.html

https://mail.python.org/pipermail/python-ideas/2011-January/009037.html

https://mail.python.org/pipermail/python-ideas/2013-February/019690.html

https://mail.python.org/pipermail/python-ideas/2013-May/020727.html

https://mail.python.org/pipermail/python-ideas/2014-March/027225.html

http://bugs.python.org/issue16276

http://bugs.python.org/issue16553

http://bugs.python.org/issue19026

http://bugs.python.org/issue5397#msg82972

[4]
https://mail.python.org/pipermail/python-dev/2007-February/071310.html
[5] (1,2)
https://mail.python.org/pipermail/python-dev/2012-December/123028.html

https://mail.python.org/pipermail/python-dev/2013-May/126327.html

[6]
https://mail.python.org/pipermail/python-dev/2012-December/123105.html
[7]
https://mail.python.org/pipermail/python-dev/2013-May/126327.html
[8]
https://mail.python.org/pipermail/python-ideas/2009-April/004163.html
[9]
https://mail.python.org/pipermail/python-dev/2013-May/126404.html
[10]
http://bugs.python.org/issue16991
[11]
https://mail.python.org/pipermail/python-dev/2012-December/123100.html

Copyright

This document has been placed in the public domain.


Source:https://github.com/python/peps/blob/main/peps/pep-0468.rst

Last modified:2025-02-01 08:59:27 GMT


[8]ページ先頭

©2009-2025 Movatter.jp