Important
This PEP is a historical document: seeUnpack for keyword arguments for up-to-date specs and documentation. Canonical typing specs are maintained at thetyping specs site; runtime typing behaviour is described in the CPython documentation.
×
See thetyping specification update process for how to propose changes to the typing spec.
Currently**kwargs can be type hinted as long as all of the keywordarguments specified by them are of the same type. However, that behaviour canbe very limiting. Therefore, in this PEP we propose a new way to enable moreprecise**kwargs typing. The new approach revolves around usingTypedDict to type**kwargs that comprise keyword arguments of differenttypes.
Currently annotating**kwargs with a typeT means that thekwargstype is in factdict[str,T]. For example:
deffoo(**kwargs:str)->None:...
means that all keyword arguments infoo are strings (i.e.,kwargs isof typedict[str,str]). This behaviour limits the ability to typeannotate**kwargs only to the cases where all of them are of the same type.However, it is often the case that keyword arguments conveyed by**kwargshave different types that are dependent on the keyword’s name. In those casestype annotating**kwargs is not possible. This is especially a problem foralready existing codebases where the need of refactoring the code in order tointroduce proper type annotations may be considered not worth the effort. Thisin turn prevents the project from getting all of the benefits that type hintingcan provide.
Moreover,**kwargs can be used to reduce the amount of code needed incases when there is a top-level function that is a part of a public API and itcalls a bunch of helper functions, all of which expect the same keywordarguments. Unfortunately, if those helper functions were to use**kwargs,there is no way to properly type hint them if the keyword arguments they expectare of different types. In addition, even if the keyword arguments are of thesame type, there is no way to check whether the function is being called withkeyword names that it actually expects.
As described in theIntended Usage section,using**kwargs is not always the best tool for the job. Despite that, it isstill a widely used pattern. As a consequence, there has been a lot ofdiscussion around supporting more precise**kwargs typing and it became afeature that would be valuable for a large part of the Python community. Thisis best illustrated by themypy GitHub issue 4441 whichcontains a lot of real world cases that could benefit from this propsal.
One more use case worth mentioning for which**kwargs are also convenient,is when a function should accommodate optional keyword-only arguments thatdon’t have default values. A need for a pattern like that can arise when valuesthat are usually used as defaults to indicate no user input, such asNone,can be passed in by a user and should result in a valid, non-default behavior.For example, this issuecame up in the popularhttpx library.
PEP 589 introduced theTypedDict type constructor that supports dictionarytypes consisting of string keys and values of potentially different types. Afunction’s keyword arguments represented by a formal parameter that begins withdouble asterisk, such as**kwargs, are received as a dictionary.Additionally, such functions are often called using unpacked dictionaries toprovide keyword arguments. This makesTypedDict a perfect candidate to beused for more precise**kwargs typing. In addition, withTypedDictkeyword names can be taken into account during static type analysis. However,specifying**kwargs type with aTypedDict means, as mentioned earlier,that each keyword argument specified by**kwargs is aTypedDict itself.For instance:
classMovie(TypedDict):name:stryear:intdeffoo(**kwargs:Movie)->None:...
means that each keyword argument infoo is itself aMovie dictionarythat has aname key with a string type value and ayear key with aninteger type value. Therefore, in order to support specifyingkwargs typeas aTypedDict without breaking current behaviour, a new construct has tobe introduced.
To support this use case, we propose reusingUnpack whichwas initially introduced inPEP 646. There are several reasons for doing so:
**kwargs typing use caseas our intention is to “unpack” the keywords arguments from the suppliedTypedDict.*args would be extended to**kwargsand those are supposed to behave similarly.Unpack for the purposes described in this PEP does notinterfere with the use cases described inPEP 646.WithUnpack we introduce a new way of annotating**kwargs.Continuing the previous example:
deffoo(**kwargs:Unpack[Movie])->None:...
would mean that the**kwargs comprise two keyword arguments specified byMovie (i.e. aname keyword of typestr and ayear keyword oftypeint). This indicates that the function should be called as follows:
kwargs:Movie={"name":"Life of Brian","year":1979}foo(**kwargs)# OK!foo(name="The Meaning of Life",year=1983)# OK!
WhenUnpack is used, type checkers treatkwargs inside thefunction body as aTypedDict:
deffoo(**kwargs:Unpack[Movie])->None:assert_type(kwargs,Movie)# OK!
Using the new annotation will not have any runtime effect - it should only betaken into account by type checkers. Any mention of errors in the followingsections relates to type checker errors.
Passing a dictionary of typedict[str,object] as a**kwargs argumentto a function that has**kwargs annotated withUnpack must generate atype checker error. On the other hand, the behaviour for functions usingstandard, untyped dictionaries can depend on the type checker. For example:
deffoo(**kwargs:Unpack[Movie])->None:...movie:dict[str,object]={"name":"Life of Brian","year":1979}foo(**movie)# WRONG! Movie is of type dict[str, object]typed_movie:Movie={"name":"The Meaning of Life","year":1983}foo(**typed_movie)# OK!another_movie={"name":"Life of Brian","year":1979}foo(**another_movie)# Depends on the type checker.
ATypedDict that is used to type**kwargs could potentially containkeys that are already defined in the function’s signature. If the duplicatename is a standard parameter, an error should be reported by type checkers.If the duplicate name is a positional-only parameter, no errors should begenerated. For example:
deffoo(name,**kwargs:Unpack[Movie])->None:...# WRONG! "name" will# always bind to the# first parameter.deffoo(name,/,**kwargs:Unpack[Movie])->None:...# OK! "name" is a# positional-only parameter,# so **kwargs can contain# a "name" keyword.
By default all keys in aTypedDict are required. This behaviour can beoverridden by setting the dictionary’stotal parameter asFalse.Moreover,PEP 655 introduced new type qualifiers -typing.Required andtyping.NotRequired - that enable specifying whether a particular key isrequired or not:
classMovie(TypedDict):title:stryear:NotRequired[int]
When using aTypedDict to type**kwargs all of the required andnon-required keys should correspond to required and non-required functionkeyword parameters. Therefore, if a required key is not supported by thecaller, then an error must be reported by type checkers.
Assignments of a function typed with**kwargs:Unpack[Movie] andanother callable type should pass type checking only if they are compatible.This can happen for the scenarios described below.
**kwargsBoth destination and source functions have a**kwargs:Unpack[TypedDict]parameter and the destination function’sTypedDict is assignable to thesource function’sTypedDict and the rest of the parameters arecompatible:
classAnimal(TypedDict):name:strclassDog(Animal):breed:strdefaccept_animal(**kwargs:Unpack[Animal]):...defaccept_dog(**kwargs:Unpack[Dog]):...accept_dog=accept_animal# OK! Expression of type Dog can be# assigned to a variable of type Animal.accept_animal=accept_dog# WRONG! Expression of type Animal# cannot be assigned to a variable of type Dog.
**kwargs and destination doesn’tThe destination callable doesn’t contain**kwargs, the source callablecontains**kwargs:Unpack[TypedDict] and the destination function’s keywordarguments are assignable to the corresponding keys in source function’sTypedDict. Moreover, not required keys should correspond to optionalfunction arguments, whereas required keys should correspond to requiredfunction arguments. Again, the rest of the parameters have to be compatible.Continuing the previous example:
classExample(TypedDict):animal:Animalstring:strnumber:NotRequired[int]defsrc(**kwargs:Unpack[Example]):...defdest(*,animal:Dog,string:str,number:int=...):...dest=src# OK!
It is worth pointing out that the destination function’s parameters that are tobe compatible with the keys and values from theTypedDict must be keywordonly:
defdest(dog:Dog,string:str,number:int=...):...dog:Dog={"name":"Daisy","breed":"labrador"}dest(dog,"some string")# OK!dest=src# Type checker error!dest(dog,"some string")# The same call fails at# runtime now because 'src' expects# keyword arguments.
The reverse situation where the destination callable contains**kwargs:Unpack[TypedDict] and the source callable doesn’t contain**kwargs should be disallowed. This is because, we cannot be sure thatadditional keyword arguments are not being passed in when an instance of asubclass had been assigned to a variable with a base class type and thenunpacked in the destination callable invocation:
defdest(**kwargs:Unpack[Animal]):...defsrc(name:str):...dog:Dog={"name":"Daisy","breed":"Labrador"}animal:Animal=dogdest=src# WRONG!dest(**animal)# Fails at runtime.
Similar situation can happen even without inheritance as compatibilitybetweenTypedDicts is based on structural subtyping.
**kwargsThe destination callable contains**kwargs:Unpack[TypedDict] and thesource callable contains untyped**kwargs:
defsrc(**kwargs):...defdest(**kwargs:Unpack[Movie]):...dest=src# OK!
**kwargs:TThe destination callable contains**kwargs:Unpack[TypedDict], the sourcecallable contains traditionally typed**kwargs:T and each of thedestination functionTypedDict’s fields is assignable to a variable oftypeT:
classVehicle:...classCar(Vehicle):...classMotorcycle(Vehicle):...classVehicles(TypedDict):car:Carmoto:Motorcycledefdest(**kwargs:Unpack[Vehicles]):...defsrc(**kwargs:Vehicle):...dest=src# OK!
On the other hand, if the destination callable contains either untyped ortraditionally typed**kwargs:T and the source callable is typed using**kwargs:Unpack[TypedDict] then an error should be generated, becausetraditionally typed**kwargs aren’t checked for keyword names.
To summarize, function parameters should behave contravariantly and functionreturn types should behave covariantly.
A previous pointmentions the problem of possibly passing additional keyword arguments byassigning a subclass instance to a variable that has a base class type. Let’sconsider the following example:
classAnimal(TypedDict):name:strclassDog(Animal):breed:strdeftakes_name(name:str):...dog:Dog={"name":"Daisy","breed":"Labrador"}animal:Animal=dogdeffoo(**kwargs:Unpack[Animal]):print(kwargs["name"].capitalize())defbar(**kwargs:Unpack[Animal]):takes_name(**kwargs)defbaz(animal:Animal):takes_name(**animal)defspam(**kwargs:Unpack[Animal]):baz(kwargs)foo(**animal)# OK! foo only expects and uses keywords of 'Animal'.bar(**animal)# WRONG! This will fail at runtime because 'breed' keyword# will be passed to 'takes_name' as well.spam(**animal)# WRONG! Again, 'breed' keyword will be eventually passed# to 'takes_name'.
In the example above, the call tofoo will not cause any issues atruntime. Even thoughfoo expectskwargs of typeAnimal it doesn’tmatter if it receives additional arguments because it only reads and uses whatit needs completely ignoring any additional values.
The calls tobar andspam will fail because an unexpected keywordargument will be passed to thetakes_name function.
Therefore,kwargs hinted with an unpackedTypedDict can only be passedto another function if the function to which unpacked kwargs are being passedto has**kwargs in its signature as well, because then additional keywordswould not cause errors at runtime during function invocation. Otherwise, thetype checker should generate an error.
In cases similar to thebar function above the problem could be workedaround by explicitly dereferencing desired fields and using them as argumentsto perform the function call:
defbar(**kwargs:Unpack[Animal]):name=kwargs["name"]takes_name(name)
Unpack with types other thanTypedDictAs described in theRationale section,TypedDict is the most natural candidate for typing**kwargs.Therefore, in the context of typing**kwargs, usingUnpack with typesother thanTypedDict should not be allowed and type checkers shouldgenerate errors in such cases.
UnpackCurrently usingUnpack in the context oftyping is interchangeable with using the asterisk syntax:
>>>Unpack[Movie]*<class '__main__.Movie'>
Therefore, in order to be compatible with the new use case,Unpack’srepr should be changed to simplyUnpack[T].
The intended use cases for this proposal are described in theMotivation section. In summary, more precise**kwargs typingcan bring benefits to already existing codebases that decided to use**kwargs initially, but now are mature enough to use a stricter contractvia type hints. Using**kwargs can also help in reducing code duplicationand the amount of copy-pasting needed when there is a bunch of functions thatrequire the same set of keyword arguments. Finally,**kwargs are useful forcases when a function needs to facilitate optional keyword arguments that don’thave obvious default values.
However, it has to be pointed out that in some cases there are better toolsfor the job than usingTypedDict to type**kwargs as proposed in thisPEP. For example, when writing new code if all the keyword arguments arerequired or have default values then writing everything explicitly is betterthan using**kwargs and aTypedDict:
deffoo(name:str,year:int):...# Preferred way.deffoo(**kwargs:Unpack[Movie]):...
Similarly, when type hinting third party libraries via stubs it is again betterto state the function signature explicitly - this is the only way to type sucha function if it has default arguments. Another issue that may arise in thiscase when trying to type hint the function with aTypedDict is that somestandard function parameters may be treated as keyword only:
deffoo(name,year):...# Function in a third party library.deffoo(Unpack[Movie]):...# Function signature in a stub file.foo("Life of Brian",1979)# This would be now failing type# checking but is fine.foo(name="Life of Brian",year=1979)# This would be the only way to call# the function now that passes type# checking.
Therefore, in this case it is again preferred to type hint such functionexplicitly as:
deffoo(name:str,year:int):...
Also, for the benefit of IDEs and documentation pages, functions that are partof the public API should prefer explicit keyword parameters whenever possible.
This PEP could be linked in thetyping module’s documentation. Moreover, anew section on usingUnpack could be added to the aforementioned docs.Similar sections could be also added to themypy documentation and thetyping documentation.
Themypy type checker alreadysupports more precise**kwargs typing usingUnpack.
Pyright type checker alsoprovides provisional supportforthis feature.
TypedDict unionsIt is possible to create unions of typed dictionaries. However, supportingtyping**kwargs with a union of typed dicts would greatly increase thecomplexity of the implementation of this PEP and there seems to be nocompelling use case to justify the support for this. Therefore, using unions oftyped dictionaries to type**kwargs as described in the context of this PEPcan result in an error:
classBook(TypedDict):genre:strpages:intTypedDictUnion=Movie|Bookdeffoo(**kwargs:Unpack[TypedDictUnion])->None:...# WRONG! Unsupported use# of a union of# TypedDicts to type# **kwargs
Instead, a function that expects a union ofTypedDicts can beoverloaded:
@overloaddeffoo(**kwargs:Unpack[Movie]):...@overloaddeffoo(**kwargs:Unpack[Book]):...
**kwargs annotationsOne way to achieve the purpose of this PEP would be to change themeaning of**kwargs annotations, so that the annotations wouldapply to the entire**kwargs dict, not to individual elements.For consistency, we would have to make an analogous change to*argsannotations.
This idea was discussed in a meeting of the typing community, and theconsensus was that the change would not be worth the cost. There is noclear migration path, the current meaning of*args and**kwargsannotations is well-established in the ecosystem, and type checkerswould have to introduce new errors for code that is currently legal.
In the previous versions of this PEP, using a double asterisk syntax wasproposed to support more precise**kwargs typing. Using this syntax,functions could be annotated as follows:
deffoo(**kwargs:**Movie):...
Which would have the same meaning as:
deffoo(**kwargs:Unpack[Movie]):...
This greatly increased the scope of the PEP, as it would require a grammarchange and adding a new dunder for theUnpack special form. At the samethe justification for introducing a new syntax was not strong enough andbecame a blocker for the whole PEP. Therefore, we decided to abandon the ideaof introducing a new syntax as a part of this PEP and may propose it again in aseparate one.
This document is placed in the public domain or under theCC0-1.0-Universal license, whichever is more permissive.
Source:https://github.com/python/peps/blob/main/peps/pep-0692.rst
Last modified:2025-03-05 16:28:34 GMT