Callables¶
Terminology¶
In this section, and throughout this specification, the term “parameter”refers to a named symbol associated with a function that receives the value ofan argument (or multiple arguments) passed to the function. The term“argument” refers to a value passed to a function when it is called.
Python supports five kinds of parameters: positional-only, keyword-only,standard (positional or keyword), variadic positional (*args), andvariadic keyword (**kwargs). Positional-only parameters can accept onlypositional arguments, and keyword-only parameters can accept only keywordarguments. Standard parameters can accept either positional or keywordarguments. Parameters of the form*args and**kwargs are variadicand accept zero or more positional or keyword arguments, respectively.
In the example below,a is a positional-only parameter,b isa standard (positional or keyword) parameter,c is a keyword-only parameter,args is a variadic parameter that accepts additional positional arguments,andkwargs is a variadic parameter that accepts additional keywordarguments:
deffunc(a:str,/,b,*args,c=0,**kwargs)->None:...
A function’s “signature” refers to its list of parameters (includingthe name, kind, optional declared type, and whether it has a defaultargument value) plus its return type. The signature of the function above is(a:str,/,b,*args,c=...,**kwargs)->None. Note that the defaultargument value for parameterc is denoted as... here because thepresence of a default value is considered part of the signature, but thespecific value is not.
The term “input signature” is used to refer to only the parameters of a function.In the example above, the input signature is(a:str,/,b,*args,c=...,**kwargs).
Positional-only parameters¶
Within a function signature, positional-only parameters are separated fromnon-positional-only parameters by a single forward slash (‘/’). Thisforward slash does not represent a parameter, but rather a delimiter. In thisexample,a is a positional-only parameter andb is a standard(positional or keyword) parameter:
deffunc(a:int,/,b:int)->None:...func(1,2)# OKfunc(1,b=2)# OKfunc(a=1,b=2)# Error
Support for the/ delimiter was introduced in Python 3.8 (PEP 570).For compatibility with earlier versions of Python, the type system alsosupports specifying positional-only parameters using adouble leadingunderscore.
Default argument values¶
In certain cases, it may be desirable to omit the default argument value fora parameter. Examples include function definitions in stub files or methodswithin a protocol or abstract base class. In such cases, the default valuemay be given as an ellipsis. For example:
deffunc(x:AnyStr,y:AnyStr=...)->AnyStr:...
If a non-ellipsis default value is present and its type can be staticallyevaluated, a type checker should verify that this type isassignable tothe declared parameter’s type:
deffunc(x:int=0):...# OKdeffunc(x:int|None=None):...# OKdeffunc(x:int=0.0):...# Errordeffunc(x:int=None):...# Error
Annotating*args and**kwargs¶
At runtime, the type of a variadic positional parameter (*args) is atuple, and the type of a variadic keyword parameter (**kwargs) is adict. However, when annotating these parameters, the type annotationrefers to the type of items within thetuple ordict (unlessUnpack is used).
Therefore, the definition:
deffunc(*args:str,**kwargs:int):...
means that the function accepts an arbitrary number of positional argumentsof typestr and an arbitrary number of keyword arguments of typeint.For example, all of the following represent function calls with validarguments:
func('a','b','c')func(x=1,y=2)func('',z=0)
In the body of functionfunc, the type of parameterargs istuple[str,...], and the type of parameterkwargs isdict[str,int].
Unpack for keyword arguments¶
typing.Unpack has two use cases in the type system:
As introduced byPEP 646, a backward-compatible form for certain operationsinvolving variadic generics. See the section on
TypeVarTuplefor details.As introduced byPEP 692, a way to annotate the
**kwargsof a function.
This second usage is described in this section. The following example:
fromtypingimportTypedDict,UnpackclassMovie(TypedDict):name:stryear:intdeffoo(**kwargs:Unpack[Movie])->None:...
means that the**kwargs comprise two keyword arguments specified byMovie (i.e. aname keyword of typestr and ayear keyword oftypeint). This indicates that the function should be called as follows:
kwargs:Movie={"name":"Life of Brian","year":1979}foo(**kwargs)# OK!foo(name="The Meaning of Life",year=1983)# OK!
WhenUnpack is used, type checkers treatkwargs inside thefunction body as aTypedDict:
deffoo(**kwargs:Unpack[Movie])->None:assert_type(kwargs,Movie)# OK!
Function calls with standard dictionaries¶
Passing a dictionary of typedict[str,object] as a**kwargs argumentto a function that has**kwargs annotated withUnpack must generate atype checker error. On the other hand, the behavior for functions usingstandard, untyped dictionaries can depend on the type checker. For example:
deffunc(**kwargs:Unpack[Movie])->None:...movie:dict[str,object]={"name":"Life of Brian","year":1979}func(**movie)# WRONG! Movie is of type dict[str, object]typed_movie:Movie={"name":"The Meaning of Life","year":1983}func(**typed_movie)# OK!another_movie={"name":"Life of Brian","year":1979}func(**another_movie)# Depends on the type checker.
Keyword collisions¶
ATypedDict that is used to type**kwargs could potentially containkeys that are already defined in the function’s signature. If the duplicatename is a standard parameter, an error should be reported by type checkers.If the duplicate name is a positional-only parameter, no errors should begenerated. For example:
deffoo(name,**kwargs:Unpack[Movie])->None:...# WRONG! "name" will# always bind to the# first parameter.deffoo(name,/,**kwargs:Unpack[Movie])->None:...# OK! "name" is a# positional-only parameter,# so **kwargs can contain# a "name" keyword.
Required and non-required items¶
Items in a TypedDict may be eitherrequired ornon-required.When using aTypedDict to type**kwargs all of the required andnon-required keys should correspond to required and non-required functionkeyword parameters. Therefore, if a required key is not provided by thecaller, then an error must be reported by type checkers.
Read-only items¶
TypedDict items may also beread-only. Marking one or more of the items of a TypedDictused to type**kwargs as read-only will have no effect on the type signature of the method.However, itwill prevent the item from being modified in the body of the function:
classArgs(TypedDict):key1:intkey2:strclassReadOnlyArgs(TypedDict):key1:ReadOnly[int]key2:ReadOnly[str]classFunction(Protocol):def__call__(self,**kwargs:Unpack[Args])->None:...defimpl(**kwargs:Unpack[ReadOnlyArgs])->None:kwargs["key1"]=3# Type check error: key1 is readonlyfn:Function=impl# Accepted by type checker: function signatures are identical
Extra items¶
If the TypedDict used for annotating**kwargs is defined to allowextra items, arbitrary additional keyword arguments of the righttype may be passed to the function:
classMovieNoExtra(TypedDict):name:strclassMovieExtra(TypedDict,extra_items=int):name:strdeff(**kwargs:Unpack[MovieNoExtra])->None:...defg(**kwargs:Unpack[MovieExtra])->None:...# Should be equivalent to:deff(*,name:str)->None:...defg(*,name:str,**kwargs:int)->None:...f(name="No Country for Old Men",year=2007)# Not OK. Unrecognized itemg(name="No Country for Old Men",year=2007)# OK
Assignment¶
Assignments of a function typed with**kwargs:Unpack[Movie] and anothercallable type should pass type checking only for the scenarios described below.
Source and destination contain**kwargs¶
Both destination and source functions have a**kwargs:Unpack[TypedDict]parameter and the destination function’sTypedDict isassignable tothe source function’sTypedDict and the rest of the parameters areassignable:
classAnimal(TypedDict):name:strclassDog(Animal):breed:strdefaccept_animal(**kwargs:Unpack[Animal]):...defaccept_dog(**kwargs:Unpack[Dog]):...accept_dog=accept_animal# OK! Expression of type Dog can be# assigned to a variable of type Animal.accept_animal=accept_dog# WRONG! Expression of type Animal# cannot be assigned to a variable of type Dog.
Source contains**kwargs and destination doesn’t¶
The destination callable doesn’t contain**kwargs, the source callablecontains**kwargs:Unpack[TypedDict] and the destination function’s keywordarguments areassignable to the corresponding keys in source function’sTypedDict. Moreover, not required keys should correspond to optionalfunction arguments, whereas required keys should correspond to requiredfunction arguments. Again, the rest of the parameters have to be assignable.Continuing the previous example:
classExample(TypedDict):animal:Animalstring:strnumber:NotRequired[int]defsrc(**kwargs:Unpack[Example]):...defdest(*,animal:Dog,string:str,number:int=...):...dest=src# OK!
It is worth pointing out that the destination function’s parameters that are tobe assignable to the keys and values from theTypedDict must be keywordonly:
defdest(dog:Dog,string:str,number:int=...):...dog:Dog={"name":"Daisy","breed":"labrador"}dest(dog,"some string")# OK!dest=src# Type checker error!dest(dog,"some string")# The same call fails at# runtime now because 'src' expects# keyword arguments.
The reverse situation where the destination callable contains**kwargs:Unpack[TypedDict] and the source callable doesn’t contain**kwargs should be disallowed. This is because, we cannot be sure thatadditional keyword arguments are not being passed in when an instance of asubclass had been assigned to a variable with a base class type and thenunpacked in the destination callable invocation:
defdest(**kwargs:Unpack[Animal]):...defsrc(name:str):...dog:Dog={"name":"Daisy","breed":"Labrador"}animal:Animal=dogdest=src# WRONG!dest(**animal)# Fails at runtime.
A similar situation can happen even without inheritance asassignability betweenTypedDicts isstructural.
Source contains untyped**kwargs¶
The destination callable contains**kwargs:Unpack[TypedDict] and thesource callable contains untyped**kwargs:
defsrc(**kwargs):...defdest(**kwargs:Unpack[Movie]):...dest=src# OK!
Source contains traditionally typed**kwargs:T¶
The destination callable contains**kwargs:Unpack[TypedDict], the sourcecallable contains traditionally typed**kwargs:T and each of thedestination functionTypedDict’s fields isassignable to a variableof typeT:
classVehicle:...classCar(Vehicle):...classMotorcycle(Vehicle):...classVehicles(TypedDict):car:Carmoto:Motorcycledefdest(**kwargs:Unpack[Vehicles]):...defsrc(**kwargs:Vehicle):...dest=src# OK!
On the other hand, if the destination callable contains either untyped ortraditionally typed**kwargs:T and the source callable is typed using**kwargs:Unpack[TypedDict] then an error should be generated, becausetraditionally typed**kwargs aren’t checked for keyword names.
To summarize, function parameters should behave contravariantly and functionreturn types should behave covariantly.
Passing kwargs inside a function to another function¶
A previous pointmentions the problem of possibly passing additional keyword arguments byassigning a subclass instance to a variable that has a base class type. Let’sconsider the following example:
classAnimal(TypedDict):name:strclassDog(Animal):breed:strdeftakes_name(name:str):...dog:Dog={"name":"Daisy","breed":"Labrador"}animal:Animal=dogdeffoo(**kwargs:Unpack[Animal]):print(kwargs["name"].capitalize())defbar(**kwargs:Unpack[Animal]):takes_name(**kwargs)defbaz(animal:Animal):takes_name(**animal)defspam(**kwargs:Unpack[Animal]):baz(kwargs)foo(**animal)# OK! foo only expects and uses keywords of 'Animal'.bar(**animal)# WRONG! This will fail at runtime because 'breed' keyword# will be passed to 'takes_name' as well.spam(**animal)# WRONG! Again, 'breed' keyword will be eventually passed# to 'takes_name'.
In the example above, the call tofoo will not cause any issues atruntime. Even thoughfoo expectskwargs of typeAnimal it doesn’tmatter if it receives additional arguments because it only reads and uses whatit needs completely ignoring any additional values.
The calls tobar andspam will fail because an unexpected keywordargument will be passed to thetakes_name function.
Therefore,kwargs hinted with an unpackedTypedDict can only be passedto another function if the function to which unpacked kwargs are being passedto has**kwargs in its signature as well, because then additional keywordswould not cause errors at runtime during function invocation. Otherwise, thetype checker should generate an error.
In cases similar to thebar function above the problem could be workedaround by explicitly dereferencing desired fields and using them as argumentsto perform the function call:
defbar(**kwargs:Unpack[Animal]):name=kwargs["name"]takes_name(name)
UsingUnpack with types other thanTypedDict¶
TypedDict is the only permitted heterogeneous type for typing**kwargs.Therefore, in the context of typing**kwargs, usingUnpack with typesother thanTypedDict should not be allowed and type checkers shouldgenerate errors in such cases.
Callable¶
TheCallable special form can be used to specify the signature ofa function within a type expression. The syntax isCallable[[Param1Type,Param2Type],ReturnType]. For example:
fromcollections.abcimportCallabledeffunc(cb:Callable[[int],str])->None:...x:Callable[[],str]
Parameters specified usingCallable are assumed to be positional-only.TheCallable form provides no way to specify keyword-only parameters,variadic parameters, or default argument values. For these use cases, seethe section onCallback protocols.
Meaning of... inCallable¶
TheCallable special form supports the use of... in place of the listof parameter types. This is agradual form indicating that the type isconsistent with any input signature:
cb1:Callable[...,str]cb1=lambdax:str(x)# OKcb1=lambda:""# OKcb2:Callable[[],str]=cb1# OK
A... can also be used withConcatenate. In this case, the parametersprior to the... are required to be present in the input signature andbe assignable, but any additional parameters are permitted:
cb3:Callable[Concatenate[int,...],str]cb3=lambdax:str(x)# OKcb3=lambdaa,b,c:str(a)# OKcb3=lambda:""# Errorcb3=lambda*,a:str(a)# Error
If the input signature in a function definition includes both a*args and**kwargs parameter and both are typed asAny (explicitly or implicitlybecause it has no annotation), a type checker should treat this as theequivalent of.... Any other parameters in the signature are unaffectedand are retained as part of the signature:
classProto1(Protocol):def__call__(self,*args:Any,**kwargs:Any)->None:...classProto2(Protocol):def__call__(self,a:int,/,*args,**kwargs)->None:...classProto3(Protocol):def__call__(self,a:int,*args:Any,**kwargs:Any)->None:...classProto4[**P](Protocol):def__call__(self,a:int,*args:P.args,**kwargs:P.kwargs)->None:...deffunc(p1:Proto1,p2:Proto2,p3:Proto3):assert_type(p1,Callable[...,None])# OKassert_type(p2,Callable[Concatenate[int,...],None])# OKassert_type(p3,Callable[...,None])# Errorassert_type(p3,Proto4[...])# OKclassA:defmethod(self,a:int,/,*args:Any,k:str,**kwargs:Any)->None:passclassB(A):# This override is OK because it is assignable to the parent's method.defmethod(self,a:float,/,b:int,*,k:str,m:str)->None:pass
The... syntax can also be used to provide aspecialized value for aParamSpec in a generic class or type alias.For example:
typeCallback[**P]=Callable[P,str]deffunc(cb:Callable[[],str])->None:f:Callback[...]=cb# OK
If... is used with signature concatenation, the... portion continuesto beconsistent with any input parameters:
typeCallbackWithInt[**P]=Callable[Concatenate[int,P],str]typeCallbackWithStr[**P]=Callable[Concatenate[str,P],str]deffunc(cb:Callable[[int,str],str])->None:f1:Callable[Concatenate[int,...],str]=cb# OKf2:Callable[Concatenate[str,...],str]=cb# Errorf3:CallbackWithInt[...]=cb# OKf4:CallbackWithStr[...]=cb# Error
Callback protocols¶
Protocols can be used to define flexible callback types that are impossible toexpress using theCallable special form as specifiedabove.This includes keyword parameters, variadic parameters, default argument values,and overloads. They can be defined as protocols with a__call__ member:
fromtypingimportProtocolclassCombiner(Protocol):def__call__(self,*args:bytes,max_len:int|None=None)->list[bytes]:...defgood_cb(*args:bytes,max_len:int|None=None)->list[bytes]:...defbad_cb(*args:bytes,max_items:int|None)->list[bytes]:...comb:Combiner=good_cb# OKcomb=bad_cb# Error! Argument 2 is not assignable because of# different parameter name and kind in the callback
Callback protocols andCallable[...] types can generally be usedinterchangeably.
Assignability rules for callables¶
A callable typeB isassignable to a callable typeA if thereturn type ofB is assignable to the return type ofA and the inputsignature ofB accepts all possible combinations of arguments that theinput signature ofA accepts. All of the specific assignability rulesdescribed below derive from this general rule.
Parameter types¶
Callable types are covariant with respect to their return types butcontravariant with respect to their parameter types. This means a callableB isassignable to callableA if the types of the parameters ofA are assignable to the parameters ofB. For example,(x:float)->int is assignable to(x:int)->float:
deffunc(cb:Callable[[float],int]):f1:Callable[[int],float]=cb# OK
Parameter kinds¶
CallableB isassignable to callableA only if all keyword-onlyparameters inA are present inB as either keyword-only parameters orstandard (positional or keyword) parameters. For example,(a:int)->Noneis assignable to(*,a:int)->None, but the converse is not true. Theorder of keyword-only parameters is ignored for purposes of assignability:
classKwOnly(Protocol):def__call__(self,*,b:int,a:int)->None:...classStandard(Protocol):def__call__(self,a:int,b:int)->None:...deffunc(standard:Standard,kw_only:KwOnly):f1:KwOnly=standard# OKf2:Standard=kw_only# Error
Likewise, callableB is assignable to callableA only if allpositional-only parameters inA are present inB as eitherpositional-only parameters or standard (positional or keyword) parameters. Thenames of positional-only parameters are ignored for purposes of assignability:
classPosOnly(Protocol):def__call__(self,not_a:int,/)->None:...classStandard(Protocol):def__call__(self,a:int)->None:...deffunc(standard:Standard,pos_only:PosOnly):f1:PosOnly=standard# OKf2:Standard=pos_only# Error
*args parameters¶
If a callableA has a signature with a*args parameter, callableBmust also have a*args parameter to beassignable toA, and thetype ofA’s*args parameter must be assignable toB’s*argsparameter:
classNoArgs(Protocol):def__call__(self)->None:...classIntArgs(Protocol):def__call__(self,*args:int)->None:...classFloatArgs(Protocol):def__call__(self,*args:float)->None:...deffunc(no_args:NoArgs,int_args:IntArgs,float_args:FloatArgs):f1:NoArgs=int_args# OKf2:NoArgs=float_args# OKf3:IntArgs=no_args# Error: missing *args parameterf4:IntArgs=float_args# OKf5:FloatArgs=no_args# Error: missing *args parameterf6:FloatArgs=int_args# Error: float is not assignable to int
If a callableA has a signature with one or more positional-onlyparameters, a callableB is assignable toA only ifB has an*args parameter whose type is assignable from the types of anyotherwise-unmatched positional-only parameters inA:
classPosOnly(Protocol):def__call__(self,a:int,b:str,/)->None:...classIntArgs(Protocol):def__call__(self,*args:int)->None:...classIntStrArgs(Protocol):def__call__(self,*args:int|str)->None:...classStrArgs(Protocol):def__call__(self,a:int,/,*args:str)->None:...classStandard(Protocol):def__call__(self,a:int,b:str)->None:...deffunc(int_args:IntArgs,int_str_args:IntStrArgs,str_args:StrArgs):f1:PosOnly=int_args# Error: str is not assignable to intf2:PosOnly=int_str_args# OKf3:PosOnly=str_args# OKf4:IntStrArgs=str_args# Error: int | str is not assignable to strf5:IntStrArgs=int_args# Error: int | str is not assignable to intf6:StrArgs=int_str_args# OKf7:StrArgs=int_args# Error: str is not assignable to intf8:IntArgs=int_str_args# OKf9:IntArgs=str_args# Error: int is not assignable to strf10:Standard=int_str_args# Error: keyword parameters a and b missingf11:Standard=str_args# Error: keyword parameter b missing
**kwargs parameters¶
If a callableA has a signature with a**kwargs parameter (without anunpackedTypedDict type annotation), callableB must also have a**kwargs parameter to beassignable toA, and the type ofA’s**kwargs parameter must be assignable toB’s**kwargsparameter:
classNoKwargs(Protocol):def__call__(self)->None:...classIntKwargs(Protocol):def__call__(self,**kwargs:int)->None:...classFloatKwargs(Protocol):def__call__(self,**kwargs:float)->None:...deffunc(no_kwargs:NoKwargs,int_kwargs:IntKwargs,float_kwargs:FloatKwargs):f1:NoKwargs=int_kwargs# OKf2:NoKwargs=float_kwargs# OKf3:IntKwargs=no_kwargs# Error: missing **kwargs parameterf4:IntKwargs=float_kwargs# OKf5:FloatKwargs=no_kwargs# Error: missing **kwargs parameterf6:FloatKwargs=int_kwargs# Error: float is not assignable to int
If a callableA has a signature with one or more keyword-only parameters,a callableB is assignable toA ifB has a**kwargs parameterwhose type is assignable from the types of any otherwise-unmatched keyword-onlyparameters inA:
classKwOnly(Protocol):def__call__(self,*,a:int,b:str)->None:...classIntKwargs(Protocol):def__call__(self,**kwargs:int)->None:...classIntStrKwargs(Protocol):def__call__(self,**kwargs:int|str)->None:...classStrKwargs(Protocol):def__call__(self,*,a:int,**kwargs:str)->None:...classStandard(Protocol):def__call__(self,a:int,b:str)->None:...deffunc(int_kwargs:IntKwargs,int_str_kwargs:IntStrKwargs,str_kwargs:StrKwargs):f1:KwOnly=int_kwargs# Error: str is not assignable to intf2:KwOnly=int_str_kwargs# OKf3:KwOnly=str_kwargs# OKf4:IntStrKwargs=str_kwargs# Error: int | str is not assignable to strf5:IntStrKwargs=int_kwargs# Error: int | str is not assignable to intf6:StrKwargs=int_str_kwargs# OKf7:StrKwargs=int_kwargs# Error: str is not assignable to intf8:IntKwargs=int_str_kwargs# OKf9:IntKwargs=str_kwargs# Error: int is not assignable to strf10:Standard=int_str_kwargs# Error: Does not accept positional argumentsf11:Standard=str_kwargs# Error: Does not accept positional arguments
Assignability rules for callable signatures that contain a**kwargs with anunpackedTypedDict are described in the sectionabove.
Signatures with ParamSpecs¶
A signature that includes*args:P.args,**kwargs:P.kwargs is equivalentto aCallable parameterized byP:
classProtocolWithP[**P](Protocol):def__call__(self,*args:P.args,**kwargs:P.kwargs)->None:...typeTypeAliasWithP[**P]=Callable[P,None]deffunc[**P](proto:ProtocolWithP[P],ta:TypeAliasWithP[P]):# These two types are equivalentf1:TypeAliasWithP[P]=proto# OKf2:ProtocolWithP[P]=ta# OK
Default argument values¶
If a callableC has a parameterx with a default argument value andA is the same asC except thatx has no default argument, thenC isassignable toA.C is also assignable toA ifA is the same asC with parameterx removed:
classDefaultArg(Protocol):def__call__(self,x:int=0)->None:...classNoDefaultArg(Protocol):def__call__(self,x:int)->None:...classNoX(Protocol):def__call__(self)->None:...deffunc(default_arg:DefaultArg):f1:NoDefaultArg=default_arg# OKf2:NoX=default_arg# OK
Overloads¶
If a callableB is overloaded with two or more signatures, it isassignable to callableA ifat least one of the overloadedsignatures inB is assignable toA:
classOverloaded(Protocol):@overloaddef__call__(self,x:int)->int:...@overloaddef__call__(self,x:str)->str:...classIntArg(Protocol):def__call__(self,x:int)->int:...classStrArg(Protocol):def__call__(self,x:str)->str:...classFloatArg(Protocol):def__call__(self,x:float)->float:...deffunc(overloaded:Overloaded):f1:IntArg=overloaded# OKf2:StrArg=overloaded# OKf3:FloatArg=overloaded# Error
If a callableA is overloaded with two or more signatures, callableBis assignable toA ifB is assignable toall of the signatures inA:
classOverloaded(Protocol):@overloaddef__call__(self,x:int,y:str)->float:...@overloaddef__call__(self,x:str)->complex:...classStrArg(Protocol):def__call__(self,x:str)->complex:...classIntStrArg(Protocol):def__call__(self,x:int|str,y:str="")->int:...deffunc(int_str_arg:IntStrArg,str_arg:StrArg):f1:Overloaded=int_str_arg# OKf2:Overloaded=str_arg# Error