Defining generic classes¶

Here is a very simple generic class that represents a stack:

fromtypingimportTypeVar,GenericT=TypeVar('T')classStack(Generic[T]):def__init__(self)->None:# Create an empty list with items of type Tself.items:list[T]=[]defpush(self,item:T)->None:self.items.append(item)defpop(self)->T:returnself.items.pop()defempty(self)->bool:returnnotself.items

TheStack class can be used to represent a stack of any type:Stack[int],Stack[tuple[int,str]], etc.

UsingStack is similar to built-in container types, likelist:

# Construct an empty Stack[int] instancestack=Stack[int]()stack.push(2)stack.pop()+1stack.push('x')# error: Argument 1 to "push" of "Stack" has incompatible type "str"; expected "int"

When creating instances of generic classes, the type argument can usually beinferred. In cases where you explicitly specify the type argument, theconstruction of the instance will be type checked correspondingly.

classBox(Generic[T]):def__init__(self,content:T)->None:self.content=contentBox(1)# OK, inferred type is Box[int]Box[int](1)# Also OKBox[int]('some string')# error: Argument 1 to "Box" has incompatible type "str"; expected "int"

Defining subclasses of generic classes¶

User-defined generic classes and generic classes defined intypingcan be used as a base class for another class (generic or non-generic). For example:

fromtypingimportGeneric,TypeVar,Mapping,IteratorKT=TypeVar('KT')VT=TypeVar('VT')# This is a generic subclass of MappingclassMyMap(Mapping[KT,VT]):def__getitem__(self,k:KT)->VT:...def__iter__(self)->Iterator[KT]:...def__len__(self)->int:...items:MyMap[str,int]# OK# This is a non-generic subclass of dictclassStrDict(dict[str,str]):def__str__(self)->str:returnf'StrDict({super().__str__()})'data:StrDict[int,int]# error: "StrDict" expects no type arguments, but 2 givendata2:StrDict# OK# This is a user-defined generic classclassReceiver(Generic[T]):defaccept(self,value:T)->None:...# This is a generic subclass of ReceiverclassAdvancedReceiver(Receiver[T]):...

Generic can be omitted from bases if there areother base classes that include type variables, such asMapping[KT,VT]in the above example. If you includeGeneric[...] in bases, thenit should list all type variables present in other bases (or more,if needed). The order of type variables is defined by the followingrules:

• IfGeneric[...] is present, then the order of variables isalways determined by their order inGeneric[...].

• If there are noGeneric[...] in bases, then all type variablesare collected in the lexicographic order (i.e. by first appearance).

For example:

fromtypingimportGeneric,TypeVar,AnyT=TypeVar('T')S=TypeVar('S')U=TypeVar('U')classOne(Generic[T]):...classAnother(Generic[T]):...classFirst(One[T],Another[S]):...classSecond(One[T],Another[S],Generic[S,U,T]):...x:First[int,str]# Here T is bound to int, S is bound to stry:Second[int,str,Any]# Here T is Any, S is int, and U is str

Generic functions¶

Type variables can be used to define generic functions. These are functionswhere the types of the arguments or return value have some relationship:

fromtypingimportTypeVar,SequenceT=TypeVar('T')# A generic function!deffirst(seq:Sequence[T])->T:returnseq[0]

As with generic classes, the type variable can be replaced with anytype. That meansfirst can be used with any sequence type, and thereturn type is derived from the sequence item type. For example:

reveal_type(first([1,2,3]))# Revealed type is "builtins.int"reveal_type(first(['a','b']))# Revealed type is "builtins.str"

Since type variables are about describing the relationship betweentwo or more types, it’s usually not useful to have a type variableonly appear once in a function signature.

Note that for convenience, a single type variable symbol (such asT above)can be used in multiple generic functions or classes, even though the logicalscope is different in each generic function or class. In the following examplewe reuse the same type variable symbol in two generic functions; these twofunctions do not share any typing relationship to each other:

fromtypingimportTypeVar,SequenceT=TypeVar('T')deffirst(seq:Sequence[T])->T:returnseq[0]deflast(seq:Sequence[T])->T:returnseq[-1]

Variables should not have a type variable in their type unless the type variableis bound by a containing generic class, generic function or generic alias.

Generic methods and generic self¶

You can also define generic methods — just use a type variable in themethod signature that is different from the type variable(s) bound inthe class definition.

# T is the type variable bound by this classclassPairedBox(Generic[T]):def__init__(self,content:T)->None:self.content=content# S is a type variable bound only in this methoddeffirst(self,x:list[S])->S:returnx[0]defpair_with_first(self,x:list[S])->tuple[S,T]:return(x[0],self.content)box=PairedBox("asdf")reveal_type(box.first([1,2,3]))# Revealed type is "builtins.int"reveal_type(box.pair_with_first([1,2,3]))# Revealed type is "tuple[builtins.int, builtins.str]"

In particular, theself argument may also be generic, allowing amethod to return the most precise type known at the point of access.In this way, for example, you can type check a chain of settermethods:

fromtypingimportTypeVarT=TypeVar('T',bound='Shape')classShape:defset_scale(self:T,scale:float)->T:self.scale=scalereturnselfclassCircle(Shape):defset_radius(self,r:float)->'Circle':self.radius=rreturnselfclassSquare(Shape):defset_width(self,w:float)->'Square':self.width=wreturnselfcircle:Circle=Circle().set_scale(0.5).set_radius(2.7)square:Square=Square().set_scale(0.5).set_width(3.2)

Without using genericself, the last two lines could not be typechecked properly, since the return type ofset_scale would beShape, which doesn’t defineset_radius orset_width.

Other uses are factory methods, such as copy and deserialization.For class methods, you can also define genericcls, usingtype:

fromtypingimportOptional,TypeVar,TypeT=TypeVar('T',bound='Friend')classFriend:other:Optional["Friend"]=None@classmethoddefmake_pair(cls:type[T])->tuple[T,T]:a,b=cls(),cls()a.other=bb.other=areturna,bclassSuperFriend(Friend):passa,b=SuperFriend.make_pair()

Note that when overriding a method with genericself, you must eitherreturn a genericself too, or return an instance of the current class.In the latter case, you must implement this method in all future subclasses.

Note also that the type checker may not always verify that the implementation of a copyor a deserialization method returns the actual type of self. Thereforeyou may need to silence the type checker inside these methods (but not at the call site),possibly by making use of theAny type or a#type:ignore comment.

Automatic self types using typing.Self¶

Since the patterns described above are quite common, a simpler syntaxwas introduced inPEP 673.

Instead of defining a type variable and using an explicit annotationforself, you can use the special typetyping.Self. This isautomatically transformed into a type variable with the current classas the upper bound, and you don’t need an annotation forself (orcls in class methods).

Here’s what the example from the previous section looks likewhen usingtyping.Self:

fromtypingimportSelfclassFriend:other:Self|None=None@classmethoddefmake_pair(cls)->tuple[Self,Self]:a,b=cls(),cls()a.other=bb.other=areturna,bclassSuperFriend(Friend):passa,b=SuperFriend.make_pair()

This is more compact than using explicit type variables. Also, you canuseSelf in attribute annotations in addition to methods.

Variance of generic types¶

There are three main kinds of generic types with respect to subtyperelations between them: invariant, covariant, and contravariant.Assuming that we have a pair of typesAnimal andBear, andBear is a subtype ofAnimal, these are defined as follows:

• A generic classMyCovGen[T] is called covariant in type parameterT ifMyCovGen[Bear] is a subtype ofMyCovGen[Animal].This is the most intuitive form of variance.

• A generic classMyContraGen[T] is called contravariant in typeparameterT ifMyContraGen[Animal] is a subtype ofMyContraGen[Bear].

• A generic classMyInvGen[T] is called invariant inT if neitherof the above is true.

Let us illustrate this by few simple examples:

# We'll use these classes in the examples belowclassShape:...classTriangle(Shape):...classSquare(Shape):...

• Most immutable containers, such asSequence andFrozenSet are covariant.Union isalso covariant in all variables:Union[Triangle,int] isa subtype ofUnion[Shape,int].

  defcount_lines(shapes:Sequence[Shape])->int:returnsum(shape.num_sidesforshapeinshapes)triangles:Sequence[Triangle]count_lines(triangles)# OKdeffoo(triangle:Triangle,num:int):shape_or_number:Union[Shape,int]# a Triangle is a Shape, and a Shape is a valid Union[Shape, int]shape_or_number=triangle

  Covariance should feel relatively intuitive, but contravariance and invariancecan be harder to reason about.

• Callable is an example of type that behaves contravariantlyin types of arguments. That is,Callable[[Shape],int] is a subtype ofCallable[[Triangle],int], despiteShape being a supertype ofTriangle. To understand this, consider:

  defcost_of_paint_required(triangle:Triangle,area_calculator:Callable[[Triangle],float])->float:returnarea_calculator(triangle)*DOLLAR_PER_SQ_FT# This straightforwardly worksdefarea_of_triangle(triangle:Triangle)->float:...cost_of_paint_required(triangle,area_of_triangle)# OK# But this works as well!defarea_of_any_shape(shape:Shape)->float:...cost_of_paint_required(triangle,area_of_any_shape)# OK

  cost_of_paint_required needs a callable that can calculate the area of atriangle. If we give it a callable that can calculate the area of anarbitrary shape (not just triangles), everything still works.

• List is an invariant generic type. Naively, one would thinkthat it is covariant, likeSequence above, but consider this code:

  classCircle(Shape):# The rotate method is only defined on Circle, not on Shapedefrotate(self):...defadd_one(things:list[Shape])->None:things.append(Shape())my_circles:list[Circle]=[]add_one(my_circles)# This may appear safe, but...my_circles[-1].rotate()# ...this will fail, since my_circles[0] is now a Shape, not a Circle

  Another example of an invariant type isDict. Most mutable containersare invariant.

By default, all user-defined generics are invariant.To declare a given generic class as covariant or contravariant usetype variables defined with special keyword argumentscovariant orcontravariant. For example:

fromtypingimportGeneric,TypeVarT_co=TypeVar('T_co',covariant=True)classBox(Generic[T_co]):# this type is declared covariantdef__init__(self,content:T_co)->None:self._content=contentdefget_content(self)->T_co:returnself._contentdeflook_into(box:Box[Animal]):...my_box=Box(Cat())look_into(my_box)# OK, but would be an error if Box was invariant in T

Type variables with upper bounds¶

By default, a type variable can be replaced with any type. This means thatyou can’t do very much with an object of typeT safely – you don’tknow anything about it!

It’s therefore often useful to be able to limit the types that a typevariable can take on, for instance, by restricting it to values that aresubtypes of a specific type.

Such a type is called the upper bound of the type variable, and is specifiedwith thebound=... keyword argument toTypeVar.

fromtypingimportTypeVar,SupportsAbsT=TypeVar('T',bound=SupportsAbs[float])

In the definition of a generic function that uses such a type variableT, the type represented byT is assumed to be a subtype ofits upper bound, so the function can use methods of the upper bound onvalues of typeT.

deflargest_in_absolute_value(*xs:T)->T:returnmax(xs,key=abs)# Okay, because T is a subtype of SupportsAbs[float].

In a call to such a function, the typeT must be replaced by atype that is a subtype of its upper bound. Continuing the exampleabove:

largest_in_absolute_value(-3.5,2)# OK, has type floatlargest_in_absolute_value(5+6j,7)# OK, has type complexlargest_in_absolute_value('a','b')# error: error: Value of type variable "T" of "largest_in_absolute_value" cannot be "str"

Type parameters of generic classes may also have upper bounds, whichrestrict the valid values for the type parameter in the same way.

Type variables with constraints¶

In some cases, it can be useful to restrict the values that a type variable can take toexactly a specific set of types. This feature is a little complex and shouldbe avoided if an upper bound can be made to work instead, as above.

An example is a type variable that can only have valuesstr andbytes:

fromtypingimportTypeVarAnyStr=TypeVar('AnyStr',str,bytes)

This is actually such a common type variable thatAnyStr isdefined intyping.

We can useAnyStr to define a function that can concatenatetwo strings or bytes objects, but it can’t be called with otherargument types:

fromtypingimportAnyStrdefconcat(x:AnyStr,y:AnyStr)->AnyStr:returnx+yconcat('a','b')# Okayconcat(b'a',b'b')# Okayconcat(1,2)# Error!

Importantly, this is different from a union type, since combinationsofstr andbytes are not accepted:

concat('string',b'bytes')# Error!

In this case, this is exactly what we want, since it’s not possibleto concatenate a string and a bytes object! If we tried to useUnion, the type checker would complain about this possibility:

defunion_concat(x:Union[str,bytes],y:Union[str,bytes])->Union[str,bytes]:returnx+y# Error: can't concatenate str and bytes

Another interesting special case is callingconcat() with asubtype ofstr:

classS(str):passss=concat(S('foo'),S('bar'))reveal_type(ss)# Revealed type is "builtins.str"

You may expect that the type ofss isS, but the type isactuallystr: a subtype gets promoted to one of the valid valuesfor the type variable, which in this case isstr.

This is thus subtly different frombounded quantification in languages such asJava, where the return type would beS. The way type checkers implement thisactually does exactly what we want forconcat, sinceconcat returns aninstance of exactlystr in the above example:

>>>print(type(ss))<class 'str'>

You can also use aTypeVar with a restricted set of possiblevalues when defining a generic class. For example, you can use the typePattern[AnyStr] for the return value ofre.compile(),since regular expressions can be based on a string or a bytes pattern.

A type variable may not have both a value restriction (seeType variables with upper bounds) and an upper bound.

Declaring decorators¶

Decorators are typically functions that take a function as an argument andreturn another function. Describing this behaviour in terms of types canbe a little tricky; we’ll show how you can useTypeVar and a specialkind of type variable called aparameter specification to do so.

Suppose we have the following decorator, not type annotated yet,that preserves the original function’s signature and merely prints the decorated function’s name:

defprinting_decorator(func):defwrapper(*args,**kwds):print("Calling",func)returnfunc(*args,**kwds)returnwrapper

and we use it to decorate functionadd_forty_two:

# A decorated function.@printing_decoratordefadd_forty_two(value:int)->int:returnvalue+42a=add_forty_two(3)

Sinceprinting_decorator is not type-annotated, the following won’t get type checked:

reveal_type(a)# Revealed type is "Any"add_forty_two('foo')# No type checker error :(

This is a sorry state of affairs!

Here’s how one could annotate the decorator:

fromtypingimportAny,Callable,TypeVar,castF=TypeVar('F',bound=Callable[...,Any])# A decorator that preserves the signature.defprinting_decorator(func:F)->F:defwrapper(*args,**kwds):print("Calling",func)returnfunc(*args,**kwds)returncast(F,wrapper)@printing_decoratordefadd_forty_two(value:int)->int:returnvalue+42a=add_forty_two(3)reveal_type(a)# Revealed type is "builtins.int"add_forty_two('x')# Argument 1 to "add_forty_two" has incompatible type "str"; expected "int"

This still has some shortcomings. First, we need to use the unsafecast() to convince type checkers thatwrapper() has the samesignature asfunc.

Second, thewrapper() function is not tightly type checked, althoughwrapper functions are typically small enough that this is not a bigproblem. This is also the reason for thecast() call in thereturn statement inprinting_decorator().

However, we can use a parameter specification (ParamSpec),for a more faithful type annotation:

fromtypingimportCallable,ParamSpec,TypeVarP=ParamSpec('P')T=TypeVar('T')defprinting_decorator(func:Callable[P,T])->Callable[P,T]:defwrapper(*args:P.args,**kwds:P.kwargs)->T:print("Calling",func)returnfunc(*args,**kwds)returnwrapper

Parameter specifications also allow you to describe decorators thatalter the signature of the input function:

fromtypingimportCallable,ParamSpec,TypeVarP=ParamSpec('P')T=TypeVar('T')# We reuse 'P' in the return type, but replace 'T' with 'str'defstringify(func:Callable[P,T])->Callable[P,str]:defwrapper(*args:P.args,**kwds:P.kwargs)->str:returnstr(func(*args,**kwds))returnwrapper@stringifydefadd_forty_two(value:int)->int:returnvalue+42a=add_forty_two(3)reveal_type(a)# Revealed type is "builtins.str"add_forty_two('x')# error: Argument 1 to "add_forty_two" has incompatible type "str"; expected "int"

Or insert an argument:

fromtypingimportCallable,Concatenate,ParamSpec,TypeVarP=ParamSpec('P')T=TypeVar('T')defprinting_decorator(func:Callable[P,T])->Callable[Concatenate[str,P],T]:defwrapper(msg:str,/,*args:P.args,**kwds:P.kwargs)->T:print("Calling",func,"with",msg)returnfunc(*args,**kwds)returnwrapper@printing_decoratordefadd_forty_two(value:int)->int:returnvalue+42a=add_forty_two('three',3)

fromtypingimportAny,Callable,TypeVarF=TypeVar('F',bound=Callable[...,Any])defroute(url:str)->Callable[[F],F]:...@route(url='/')defindex(request:Any)->str:return'Hello world'

fromtypingimportAny,Callable,Optional,TypeVar,overloadF=TypeVar('F',bound=Callable[...,Any])# Bare decorator usage@overloaddefatomic(__func:F)->F:...# Decorator with arguments@overloaddefatomic(*,savepoint:bool=True)->Callable[[F],F]:...# Implementationdefatomic(__func:Optional[Callable[...,Any]]=None,*,savepoint:bool=True):defdecorator(func:Callable[...,Any]):...# Code goes hereif__funcisnotNone:returndecorator(__func)else:returndecorator# Usage@atomicdeffunc1()->None:...@atomic(savepoint=False)deffunc2()->None:...

Generic protocols¶

Protocols can also be generic (see alsoProtocols and structural subtyping). Severalpredefined protocols are generic, such asIterable[T], and you can define additional genericprotocols. Generic protocols mostly follow the normal rules for generic classes.Example:

fromtypingimportProtocol,TypeVarT=TypeVar('T')classBox(Protocol[T]):content:Tdefdo_stuff(one:Box[str],other:Box[bytes])->None:...classStringWrapper:def__init__(self,content:str)->None:self.content=contentclassBytesWrapper:def__init__(self,content:bytes)->None:self.content=contentdo_stuff(StringWrapper('one'),BytesWrapper(b'other'))# OKx:Box[float]=...y:Box[int]=...x=y# Error -- Box is invariant

Note thatclassClassName(Protocol[T]) is allowed as a shorthand forclassClassName(Protocol,Generic[T]), as perPEP 544: Generic protocols,

The main difference between generic protocols and ordinary generic classes isthat the declared variances of generic type variables in a protocol are checkedagainst how they are used in the protocol definition. The protocol in thisexample is rejected, since the type variableT is used covariantly as areturn type, but the type variable is invariant:

fromtypingimportProtocol,TypeVarT=TypeVar('T')classReadOnlyBox(Protocol[T]):# error: Invariant type variable "T" used in protocol where covariant one is expecteddefcontent(self)->T:...

This example correctly uses a covariant type variable:

fromtypingimportProtocol,TypeVarT_co=TypeVar('T_co',covariant=True)classReadOnlyBox(Protocol[T_co]):# OKdefcontent(self)->T_co:...ax:ReadOnlyBox[float]=...ay:ReadOnlyBox[int]=...ax=ay# OK -- ReadOnlyBox is covariant

SeeVariance of generic types for more about variance.

Generic protocols can also be recursive. Example:

T=TypeVar('T')classLinked(Protocol[T]):val:Tdefnext(self)->'Linked[T]':...classL:val:intdefnext(self)->'L':...deflast(seq:Linked[T])->T:...result=last(L())reveal_type(result)# Revealed type is "builtins.int"

Generic type aliases¶

Type aliases can be generic. In this case they can be used in two ways:Subscripted aliases are equivalent to original types with substituted typevariables, so the number of type arguments must match the number of free type variablesin the generic type alias. Unsubscripted aliases are treated as original types with freevariables replaced withAny. Examples (followingPEP 484: Type aliases):

fromtypingimportTypeVar,Iterable,Union,CallableS=TypeVar('S')TInt=tuple[int,S]UInt=Union[S,int]CBack=Callable[...,S]defresponse(query:str)->UInt[str]:# Same as Union[str, int]...defactivate(cb:CBack[S])->S:# Same as Callable[..., S]...table_entry:TInt# Same as tuple[int, Any]T=TypeVar('T',int,float,complex)Vec=Iterable[tuple[T,T]]definproduct(v:Vec[T])->T:returnsum(x*yforx,yinv)defdilate(v:Vec[T],scale:T)->Vec[T]:return((x*scale,y*scale)forx,yinv)v1:Vec[int]=[]# Same as Iterable[tuple[int, int]]v2:Vec=[]# Same as Iterable[tuple[Any, Any]]v3:Vec[int,int]=[]# Error: Invalid alias, too many type arguments!

Type aliases can be imported from modules just like other names. Analias can also target another alias, although building complex chainsof aliases is not recommended – this impedes code readability, thusdefeating the purpose of using aliases. Example:

fromtypingimportTypeVar,Generic,Optionalfromexample1importAliasTypefromexample2importVec# AliasType and Vec are type aliases (Vec as defined above)deffun()->AliasType:...T=TypeVar('T')classNewVec(Vec[T]):...fori,jinNewVec[int]():...OIntVec=Optional[Vec[int]]

Using type variable bounds or values in generic aliases has the same effectas in generic classes/functions.

Generic class internals¶

You may wonder what happens at runtime when you index a generic class.Indexing returns ageneric alias to the original class that returns instancesof the original class on instantiation:

>>>fromtypingimportTypeVar,Generic>>>T=TypeVar('T')>>>classStack(Generic[T]):...>>>Stack__main__.Stack>>>Stack[int]__main__.Stack[int]>>>instance=Stack[int]()>>>instance.__class____main__.Stack

Generic aliases can be instantiated or subclassed, similar to realclasses, but the above examples illustrate that type variables areerased at runtime. GenericStack instances are just ordinaryPython objects, and they have no extra runtime overhead or magic dueto being generic, other than overloading the indexing operation.

Note that in Python 3.8 and lower, the built-in typeslist,dict and others do not support indexing.This is why we have the aliasesList,Dict and so on in thetypingmodule. Indexing these aliases gives you a generic alias thatresembles generic aliases constructed by directly indexing the targetclass in more recent versions of Python:

>>># Only relevant for Python 3.8 and below>>># For Python 3.9 onwards, prefer `list[int]` syntax>>>fromtypingimportList>>>List[int]typing.List[int]

Note that the generic aliases intyping don’t support constructinginstances:

>>>fromtypingimportList>>>List[int]()Traceback (most recent call last):...TypeError:Type List cannot be instantiated; use list() instead

Credits¶

This document is based on themypy documentation