Performance

Thetyping module is one of the heaviest and slowest modules inthe standard library even with all the optimizations made. Mainly this isbecause subscripted generic types (seePEP 484 for definition of terms usedin this PEP) are class objects (see also[1]). There are three main ways howthe performance can be improved with the help of the proposed special methods:

• Creation of generic classes is slow since theGenericMeta.__new__ isvery slow; we will not need it anymore.
• Very long method resolution orders (MROs) for generic classes will behalf as long; they are present because we duplicate thecollections.abcinheritance chain intyping.
• Instantiation of generic classes will be faster (this is minor however).

Metaclass conflicts

All generic types are instances ofGenericMeta, so if a user usesa custom metaclass, then it is hard to make a corresponding class generic.This is particularly hard for library classes that a user doesn’t control.A workaround is to always mix-inGenericMeta:

classAdHocMeta(GenericMeta,LibraryMeta):passclassUserClass(LibraryBase,Generic[T],metaclass=AdHocMeta):...

but this is not always practical or even possible. With the help of theproposed special attributes theGenericMeta metaclass will not be needed.

Hacks and bugs that will be removed by this proposal

• _generic_new hack that exists because__init__ is not called oninstances with a type differing from the type whose__new__ was called,C[int]().__class__isC.
• _next_in_mro speed hack will be not necessary since subscription willnot create new classes.
• Uglysys._getframe hack. This one is particularly nasty since it lookslike we can’t remove it without changes outsidetyping.
• Currently generics do dangerous things with private ABC cachesto fix large memory consumption that grows at least as O(N²),see[2]. This point is also important because it was recently proposed tore-implementABCMeta in C.
• Problems with sharing attributes between subscripted generics,see[3]. The current solution already uses__getattr__ and__setattr__,but it is still incomplete, and solving this without the current proposalwill be hard and will need__getattribute__.
• _no_slots_copy hack, where we clean up the class dictionary on everysubscription thus allowing generics with__slots__.
• General complexity of thetyping module. The new proposal will notonly allow to remove the above-mentioned hacks/bugs, but also simplifythe implementation, so that it will be easier to maintain.

__class_getitem__

The idea of__class_getitem__ is simple: it is an exact analog of__getitem__ with an exception that it is called on a class thatdefines it, not on its instances. This allows us to avoidGenericMeta.__getitem__ for things likeIterable[int].The__class_getitem__ is automatically a class method anddoes not require@classmethod decorator (similar to__init_subclass__) and is inherited like normal attributes.For example:

classMyList:def__getitem__(self,index):returnindex+1def__class_getitem__(cls,item):returnf"{cls.__name__}[{item.__name__}]"classMyOtherList(MyList):passassertMyList()[0]==1assertMyList[int]=="MyList[int]"assertMyOtherList()[0]==1assertMyOtherList[int]=="MyOtherList[int]"

Note that this method is used as a fallback, so if a metaclass defines__getitem__, then that will have the priority.

__mro_entries__

If an object that is not a class object appears in the tuple of bases ofa class definition, then method__mro_entries__ is searched on it.If found, it is called with the original tuple of bases as an argument.The result of the call must be a tuple, that is unpacked in the base classesin place of this object. (If the tuple is empty, this means that the originalbases is simply discarded.) If there are more than one object with__mro_entries__, then all of them are called with the same original tupleof bases. This step happens first in the process of creation of a class,all other steps, including checks for duplicate bases and MRO calculation,happen normally with the updated bases.

Using the method API instead of just an attribute is necessary to avoidinconsistent MRO errors, and perform other manipulations that are currentlydone byGenericMeta.__new__. The original bases are stored as__orig_bases__ in the class namespace (currently this is also done bythe metaclass). For example:

classGenericAlias:def__init__(self,origin,item):self.origin=originself.item=itemdef__mro_entries__(self,bases):return(self.origin,)classNewList:def__class_getitem__(cls,item):returnGenericAlias(cls,item)classTokens(NewList[int]):...assertTokens.__bases__==(NewList,)assertTokens.__orig_bases__==(NewList[int],)assertTokens.__mro__==(Tokens,NewList,object)

Resolution using__mro_entries__ happensonly in bases of a classdefinition statement. In all other situations where a class object isexpected, no such resolution will happen, this includesisinstanceandissubclass built-in functions.

NOTE: These two method names are reserved for use by thetyping moduleand the generic types machinery, and any other use is discouraged.The reference implementation (with tests) can be found in[4], andthe proposal was originally posted and discussed on thetyping tracker,see[5].

Dynamic class creation andtypes.resolve_bases

type.__new__ will not perform any MRO entry resolution. So that a directcalltype('Tokens',(List[int],),{}) will fail. This is done forperformance reasons and to minimize the number of implicit transformations.Instead, a helper functionresolve_bases will be added tothetypes module to allow an explicit__mro_entries__ resolution inthe context of dynamic class creation. Correspondingly,types.new_classwill be updated to reflect the new class creation steps while maintainingthe backwards compatibility:

defnew_class(name,bases=(),kwds=None,exec_body=None):resolved_bases=resolve_bases(bases)# This step is addedmeta,ns,kwds=prepare_class(name,resolved_bases,kwds)ifexec_bodyisnotNone:exec_body(ns)ns['__orig_bases__']=bases# This step is addedreturnmeta(name,resolved_bases,ns,**kwds)

Using__class_getitem__ in C extensions

As mentioned above,__class_getitem__ is automatically a class methodif defined in Python code. To define this method in a C extension, oneshould use flagsMETH_O|METH_CLASS. For example, a simple way to makean extension class generic is to use a method that simply returns theoriginal class objects, thus fully erasing the type information at runtime,and deferring all check to static type checkers only:

typedefstruct{PyObject_HEAD/*...yourcode...*/}SimpleGeneric;staticPyObject*simple_class_getitem(PyObject*type,PyObject*item){Py_INCREF(type);returntype;}staticPyMethodDefsimple_generic_methods[]={{"__class_getitem__",simple_class_getitem,METH_O|METH_CLASS,NULL},/*...othermethods...*/};PyTypeObjectSimpleGeneric_Type={PyVarObject_HEAD_INIT(NULL,0)"SimpleGeneric",sizeof(SimpleGeneric),0,.tp_flags=Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,.tp_methods=simple_generic_methods,};

Such class can be used as a normal generic in Python type annotations(a corresponding stub file should be provided for static type checkers,seePEP 484 for details):

fromsimple_extensionimportSimpleGenericfromtypingimportTypeVarT=TypeVar('T')Alias=SimpleGeneric[str,T]classSubClass(SimpleGeneric[T,int]):...data:Alias[int]# Works at runtimemore_data:SubClass[str]# Also works at runtime