Currently, customising class creation requires the use of a custom metaclass.This custom metaclass then persists for the entire lifecycle of the class,creating the potential for spurious metaclass conflicts.
This PEP proposes to instead support a wide range of customisationscenarios through a new__init_subclass__ hook in the class body,and a hook to initialize attributes.
The new mechanism should be easier to understand and use thanimplementing a custom metaclass, and thus should provide a gentlerintroduction to the full power of Python’s metaclass machinery.
Metaclasses are a powerful tool to customize class creation. They have,however, the problem that there is no automatic way to combine metaclasses.If one wants to use two metaclasses for a class, a new metaclass combiningthose two needs to be created, typically manually.
This need often occurs as a surprise to a user: inheriting from two baseclasses coming from two different libraries suddenly raises the necessityto manually create a combined metaclass, where typically one is notinterested in those details about the libraries at all. This becomeseven worse if one library starts to make use of a metaclass which ithas not done before. While the library itself continues to work perfectly,suddenly every code combining those classes with classes from another libraryfails.
While there are many possible ways to use a metaclass, the vast majorityof use cases falls into just three categories: some initialization coderunning after class creation, the initialization of descriptors andkeeping the order in which class attributes were defined.
The first two categories can easily be achieved by having simple hooksinto the class creation:
__init_subclass__ hook that initializesall subclasses of a given class.__set_name__ hook is called on all theattribute (descriptors) defined in the class, andThe third category is the topic of another PEP,PEP 520.
As an example, the first use case looks as follows:
>>>classQuestBase:...# this is implicitly a @classmethod (see below for motivation)...def__init_subclass__(cls,swallow,**kwargs):...cls.swallow=swallow...super().__init_subclass__(**kwargs)>>>classQuest(QuestBase,swallow="african"):...pass>>>Quest.swallow'african'
The base classobject contains an empty__init_subclass__method which serves as an endpoint for cooperative multiple inheritance.Note that this method has no keyword arguments, meaning that allmethods which are more specialized have to process all keywordarguments.
This general proposal is not a new idea (it was first suggested forinclusion in the language definitionmore than 10 years ago, and asimilar mechanism has long been supported byZope’s ExtensionClass),but the situation has changed sufficiently in recent years thatthe idea is worth reconsidering for inclusion.
The second part of the proposal adds an__set_name__initializer for class attributes, especially if they are descriptors.Descriptors are defined in the body of aclass, but they do not know anything about that class, they do noteven know the name they are accessed with. They do get to know theirowner once__get__ is called, but still they do not know theirname. This is unfortunate, for example they cannot put theirassociated value into their object’s__dict__ under their name,since they do not know that name. This problem has been solved manytimes, and is one of the most important reasons to have a metaclass ina library. While it would be easy to implement such a mechanism usingthe first part of the proposal, it makes sense to have one solutionfor this problem for everyone.
To give an example of its usage, imagine a descriptor representing weakreferenced values:
importweakrefclassWeakAttribute:def__get__(self,instance,owner):returninstance.__dict__[self.name]()def__set__(self,instance,value):instance.__dict__[self.name]=weakref.ref(value)# this is the new initializer:def__set_name__(self,owner,name):self.name=name
Such aWeakAttribute may, for example, be used in a tree structurewhere one wants to avoid cyclic references via the parent:
classTreeNode:parent=WeakAttribute()def__init__(self,parent):self.parent=parent
Note that theparent attribute is used like a normal attribute,yet the tree contains no cyclic references and can thus be easilygarbage collected when out of use. Theparent attribute magicallybecomesNone once the parent ceases existing.
While this example looks very trivial, it should be noted that untilnow such an attribute cannot be defined without the use of a metaclass.And given that such a metaclass can make life very hard, this kind ofattribute does not exist yet.
Initializing descriptors could simply be done in the__init_subclass__ hook. But this would mean that descriptors canonly be used in classes that have the proper hook, the generic versionlike in the example would not work generally. One could also call__set_name__ from within the base implementation ofobject.__init_subclass__. But given that it is a common mistaketo forget to callsuper(), it would happen too often that suddenlydescriptors are not initialized.
Understanding Python’s metaclasses requires a deep understanding ofthe type system and the class construction process. This is legitimatelyseen as challenging, due to the need to keep multiple moving parts (the code,the metaclass hint, the actual metaclass, the class object, instances of theclass object) clearly distinct in your mind. Even when you know the rules,it’s still easy to make a mistake if you’re not being extremely careful.
Understanding the proposed implicit class initialization hook only requiresordinary method inheritance, which isn’t quite as daunting a task. The newhook provides a more gradual path towards understanding all of the phasesinvolved in the class definition process.
One of the big issues that makes library authors reluctant to use metaclasses(even when they would be appropriate) is the risk of metaclass conflicts.These occur whenever two unrelated metaclasses are used by the desiredparents of a class definition. This risk also makes it very difficult toadd a metaclass to a class that has previously been published without one.
By contrast, adding an__init_subclass__ method to an existing type posesa similar level of risk to adding an__init__ method: technically, thereis a risk of breaking poorly implemented subclasses, but when that occurs,it is recognised as a bug in the subclass rather than the library authorbreaching backwards compatibility guarantees.
Especially when writing a plugin system, one likes to register newsubclasses of a plugin baseclass. This can be done as follows:
classPluginBase:subclasses=[]def__init_subclass__(cls,**kwargs):super().__init_subclass__(**kwargs)cls.subclasses.append(cls)
In this example,PluginBase.subclasses will contain a plain list of allsubclasses in the entire inheritance tree. One should note that this alsoworks nicely as a mixin class.
There are many designs of Python descriptors in the wild which, forexample, check boundaries of values. Often those “traits” need some supportof a metaclass to work. This is how this would look like with thisPEP:
classTrait:def__init__(self,minimum,maximum):self.minimum=minimumself.maximum=maximumdef__get__(self,instance,owner):returninstance.__dict__[self.key]def__set__(self,instance,value):ifself.minimum<value<self.maximum:instance.__dict__[self.key]=valueelse:raiseValueError("value not in range")def__set_name__(self,owner,name):self.key=name
The hooks are called in the following order:type.__new__ callsthe__set_name__ hooks on the descriptor after the new class has beeninitialized. Then it calls__init_subclass__ on the base class, onsuper(), to be precise. This means that subclass initializers alreadysee the fully initialized descriptors. This way,__init_subclass__ userscan fix all descriptors again if this is needed.
Another option would have been to call__set_name__ in the baseimplementation ofobject.__init_subclass__. This way it would be possibleeven to prevent__set_name__ from being called. Most of the times,however, such a prevention would be accidental, as it often happens that a calltosuper() is forgotten.
As a third option, all the work could have been done intype.__init__.Most metaclasses do their work in__new__, as this is recommended bythe documentation. Many metaclasses modify their arguments before theypass them over tosuper().__new__. For compatibility with those kindof classes, the hooks should be called from__new__.
Another small change should be done: in the current implementation ofCPython,type.__init__ explicitly forbids the use of keyword arguments,whiletype.__new__ allows for its attributes to be shipped as keywordarguments. This is weirdly incoherent, and thus it should be forbidden.While it would be possible to retain the current behavior, it would be betterif this was fixed, as it is probably not used at all: the only use case wouldbe that at metaclass calls itssuper().__new__ withname,bases anddict (yes,dict, notnamespace orns as mostly used with modernmetaclasses) as keyword arguments. This should not be done. This littlechange simplifies the implementation of this PEP significantly, whileimproving the coherence of Python overall.
As a second change, the newtype.__init__ just ignores keywordarguments. Currently, it insists that no keyword arguments are given. Thisleads to a (wanted) error if one gives keyword arguments to a class declarationif the metaclass does not process them. Metaclass authors that do want toaccept keyword arguments must filter them out by overriding__init__.
In the new code, it is not__init__ that complains about keyword arguments,but__init_subclass__, whose default implementation takes no arguments. Ina classical inheritance scheme using the method resolution order, each__init_subclass__ may take out it’s keyword arguments until none are left,which is checked by the default implementation of__init_subclass__.
For readers who prefer reading Python over English, this PEP proposes toreplace the currenttype andobject with the following:
classNewType(type):def__new__(cls,*args,**kwargs):iflen(args)!=3:returnsuper().__new__(cls,*args)name,bases,ns=argsinit=ns.get('__init_subclass__')ifisinstance(init,types.FunctionType):ns['__init_subclass__']=classmethod(init)self=super().__new__(cls,name,bases,ns)fork,vinself.__dict__.items():func=getattr(v,'__set_name__',None)iffuncisnotNone:func(self,k)super(self,self).__init_subclass__(**kwargs)returnselfdef__init__(self,name,bases,ns,**kwargs):super().__init__(name,bases,ns)classNewObject(object):@classmethoddef__init_subclass__(cls):pass
The reference implementation for this PEP is attached toissue 27366.
The exact calling sequence intype.__new__ is slightly changed, raisingfears of backwards compatibility. It should be assured by tests that common usecases behave as desired.
The following class definitions (except the one defining the metaclass)continue to fail with aTypeError as superfluous class arguments are passed:
classMyMeta(type):passclassMyClass(metaclass=MyMeta,otherarg=1):passMyMeta("MyClass",(),otherargs=1)importtypestypes.new_class("MyClass",(),dict(metaclass=MyMeta,otherarg=1))types.prepare_class("MyClass",(),dict(metaclass=MyMeta,otherarg=1))
A metaclass defining only a__new__ method which is interested in keywordarguments now does not need to define an__init__ method anymore, as thedefaulttype.__init__ ignores keyword arguments. This is nicely in linewith the recommendation to override__new__ in metaclasses instead of__init__. The following code does not fail anymore:
classMyMeta(type):def__new__(cls,name,bases,namespace,otherarg):returnsuper().__new__(cls,name,bases,namespace)classMyClass(metaclass=MyMeta,otherarg=1):pass
Only defining an__init__ method in a metaclass continues to fail withTypeError if keyword arguments are given:
classMyMeta(type):def__init__(self,name,bases,namespace,otherarg):super().__init__(name,bases,namespace)classMyClass(metaclass=MyMeta,otherarg=1):pass
Defining both__init__ and__new__ continues to work fine.
About the only thing that stops working is passing the arguments oftype.__new__ as keyword arguments:
classMyMeta(type):def__new__(cls,name,bases,namespace):returnsuper().__new__(cls,name=name,bases=bases,dict=namespace)classMyClass(metaclass=MyMeta):pass
This will now raiseTypeError, but this is weird code, and easyto fix even if someone used this feature.
Adding an__autodecorate__ hook that would be called on the classitself was the proposed idea ofPEP 422. Most examples work the sameway or even better if the hook is called only on strict subclasses. In general,it is much easier to arrange to explicitly call the hook on the class in which itis defined (to opt-in to such a behavior) than to opt-out (by remember to check forclsis__class in the hook body), meaning that one does not want the hook to becalled on the class it is defined in.
This becomes most evident if the class in question is designed as amixin: it is very unlikely that the code of the mixin is to beexecuted for the mixin class itself, as it is not supposed to be acomplete class on its own.
The original proposal also made major changes in the classinitialization process, rendering it impossible to back-port theproposal to older Python versions.
When it’s desired to also call the hook on the base class, two mechanisms are available:
__init_subclass__implementation. The original “base” class can then list the new mixin as itsfirst parent class.__init_subclass__.Calling__init_subclass__ explicitly from a class decorator will generally beundesirable, as this will also typically call__init_subclass__ a second time onthe parent class, which is unlikely to be desired behaviour.
Other names for the hook were presented, namely__decorate__ or__autodecorate__. This proposal opts for__init_subclass__ asit is very close to the__init__ method, just for the subclass,while it is not very close to decorators, as it does not return theclass.
For the__set_name__ hook other names have been proposed as well,__set_owner__,__set_ownership__ and__init_descriptor__.
__init_subclass__One could require the explicit use of@classmethod on the__init_subclass__ decorator. It was made implicit since there’s nosensible interpretation for leaving it out, and that case would needto be detected anyway in order to give a useful error message.
This decision was reinforced after noticing that the user experience ofdefining__prepare__ and forgetting the@classmethod methoddecorator is singularly incomprehensible (particularly sincePEP 3115documents it as an ordinary method, and the current documentation doesn’texplicitly say anything one way or the other).
__new__-like hookInPEP 422 the hook worked more like the__new__ method than the__init__ method, meaning that it returned a class instead ofmodifying one. This allows a bit more flexibility, but at the costof much harder implementation and undesired side effects.
This got its ownPEP 520.
This used to be a competing proposal toPEP 422 by Alyssa Coghlan and DanielUrban.PEP 422 intended to achieve the same goals as this PEP, but with adifferent way of implementation. In the meantime,PEP 422 has been withdrawnfavouring this approach.
This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0487.rst
Last modified:2025-02-01 08:59:27 GMT