Currently, customising class creation requires the use of a custom metaclass.This custom metaclass then persists for the entire lifecycle of the class,creating the potential for spurious metaclass conflicts.
This PEP proposes to instead support a wide range of customisationscenarios through a newnamespace parameter in the class header, anda new__autodecorate__ hook in the class body.
The new mechanism should be easier to understand and use thanimplementing a custom metaclass, and thus should provide a gentlerintroduction to the full power Python’s metaclass machinery.
This proposal has been withdrawn in favour of Martin Teichmann’s proposalinPEP 487, which achieves the same goals through a simpler, easier to use__init_subclass__ hook that simply isn’t invoked for the base classthat defines the hook.
For an already created classcls, the term “metaclass” has a clearmeaning: it is the value oftype(cls).
During class creation, it has another meaning: it is also used to refer tothe metaclass hint that may be provided as part of the class definition.While in many cases these two meanings end up referring to one and the sameobject, there are two situations where that is not the case:
type, then it isconsidered as a candidate metaclass along with the metaclasses of all ofthe parents of the class being defined. If a more appropriate metaclass isfound amongst the candidates, then it will be used instead of the onegiven in the metaclass hint.type, or, inPython 3.3 or later,types.new_class) the actual metaclass is thendetermined based on the parent classes.It is notable that only the actual metaclass is inherited - a factoryfunction used as a metaclass hook sees only the class currently beingdefined, and is not invoked for any subclasses.
In Python 3, the metaclass hint is provided using themetaclass=Metakeyword syntax in the class header. This allows the__prepare__ methodon the metaclass to be used to create thelocals() namespace used duringexecution of the class body (for example, specifying the use ofcollections.OrderedDict instead of a regulardict).
In Python 2, there was no__prepare__ method (that API was added forPython 3 byPEP 3115). Instead, a class body could set the__metaclass__attribute, and the class creation process would extract that value from theclass namespace to use as the metaclass hint. There ispublished code thatmakes use of this feature.
Another new feature in Python 3 is the zero-argument form of thesuper()builtin, introduced byPEP 3135. This feature uses an implicit__class__reference to the class being defined to replace the “by name” referencesrequired in Python 2. Just as code invoked during execution of a Python 2metaclass could not call methods that referenced the class by name (as thename had not yet been bound in the containing scope), similarly, Python 3metaclasses cannot call methods that rely on the implicit__class__reference (as it is not populated until after the metaclass has returnedcontrol to the class creation machinery).
Finally, when a class uses a custom metaclass, it can pose additionalchallenges to the use of multiple inheritance, as a new class cannotinherit from parent classes with unrelated metaclasses. This means thatit is impossible to add a metaclass to an already published class: suchan addition is a backwards incompatible change due to the risk of metaclassconflicts.
This PEP proposes that a new mechanism to customise class creation beadded to Python 3.4 that meets the following criteria:
__class__ reference andzero-argumentsuper() syntax introduced byPEP 3135__metaclass__ hookOne mechanism that can achieve this goal is to add a new implicit classdecoration hook, modelled directly on the existing explicit classdecorators, but defined in the class body or in a parent class, rather thanbeing part of the class definition header.
Specifically, it is proposed that class definitions be able to provide aclass initialisation hook as follows:
classExample:def__autodecorate__(cls):# This is invoked after the class is created, but before any# explicit decorators are called# The usual super() mechanisms are used to correctly support# multiple inheritance. The class decorator style signature helps# ensure that invoking the parent class is as simple as possible.cls=super().__autodecorate__()returncls
To simplify the cooperative multiple inheritance case,object will gaina default implementation of the hook that returns the class unmodified:
classobject:def__autodecorate__(cls):returncls
If a metaclass wishes to block implicit class decoration for some reason, itmust arrange forcls.__autodecorate__ to triggerAttributeError.
If present on the created object, this new hook will be called by the classcreation machineryafter the__class__ reference has been initialised.Fortypes.new_class(), it will be called as the last step beforereturning the created class object.__autodecorate__ is implicitlyconverted to a class method when the class is created (prior to the hookbeing invoked).
Note, that when__autodecorate__ is called, the name of the class is notyet bound to the new class object. As a consequence, the two argument formofsuper() cannot be used to call methods (e.g.,super(Example,cls)wouldn’t work in the example above). However, the zero argument form ofsuper() works as expected, since the__class__ reference is alreadyinitialised.
This general proposal is not a new idea (it was first suggested forinclusion in the language definitionmore than 10 years ago, and asimilar mechanism has long been supported byZope’s ExtensionClass),but the situation has changed sufficiently in recent years thatthe idea is worth reconsidering for inclusion as a native language feature.
In addition, the introduction of the metaclass__prepare__ method in PEP3115 allows a further enhancement that was not possible in Python 2: thisPEP also proposes thattype.__prepare__ be updated to accept a factoryfunction as anamespace keyword-only argument. If present, the valueprovided as thenamespace argument will be called without argumentsto create the result oftype.__prepare__ instead of using a freshlycreated dictionary instance. For example, the following will usean ordered dictionary as the class namespace:
classOrderedExample(namespace=collections.OrderedDict):def__autodecorate__(cls):# cls.__dict__ is still a read-only proxy to the class namespace,# but the underlying storage is an OrderedDict instance
Note
This PEP, along with the existing ability to use __prepare__ to share asingle namespace amongst multiple class objects, highlights a possibleissue with the attribute lookup caching: when the underlying mapping isupdated by other means, the attribute lookup cache is not invalidatedcorrectly (this is a key part of the reason class__dict__ attributesproduce a read-only view of the underlying storage).
Since the optimisation provided by that cache is highly desirable,the use of a preexisting namespace as the class namespace may need tobe declared as officially unsupported (since the observed behaviour israther strange when the caches get out of sync).
Currently, to use a different type (such ascollections.OrderedDict) fora class namespace, or to use a pre-populated namespace, it is necessary towrite and use a custom metaclass. With this PEP, using a custom namespacebecomes as simple as specifying an appropriate factory function in theclass header.
Understanding Python’s metaclasses requires a deep understanding ofthe type system and the class construction process. This is legitimatelyseen as challenging, due to the need to keep multiple moving parts (the code,the metaclass hint, the actual metaclass, the class object, instances of theclass object) clearly distinct in your mind. Even when you know the rules,it’s still easy to make a mistake if you’re not being extremely careful.An earlier version of this PEP actually included such a mistake: itstated “subclass of type” for a constraint that is actually “instance oftype”.
Understanding the proposed implicit class decoration hook only requiresunderstanding decorators and ordinary method inheritance, which isn’tquite as daunting a task. The new hook provides a more gradual pathtowards understanding all of the phases involved in the class definitionprocess.
One of the big issues that makes library authors reluctant to use metaclasses(even when they would be appropriate) is the risk of metaclass conflicts.These occur whenever two unrelated metaclasses are used by the desiredparents of a class definition. This risk also makes it very difficult toadd a metaclass to a class that has previously been published without one.
By contrast, adding an__autodecorate__ method to an existing type posesa similar level of risk to adding an__init__ method: technically, thereis a risk of breaking poorly implemented subclasses, but when that occurs,it is recognised as a bug in the subclass rather than the library authorbreaching backwards compatibility guarantees. In fact, due to the constrainedsignature of__autodecorate__, the risk in this case is actually evenlower than in the case of__init__.
Unlike code that runs as part of the metaclass, code that runs as part ofthe new hook will be able to freely invoke class methods that rely on theimplicit__class__ reference introduced byPEP 3135, including methodsthat use the zero argument form ofsuper().
__metaclass__For use cases that don’t involve completely replacing the defined class,Python 2 code that dynamically set__metaclass__ can now dynamicallyset__autodecorate__ instead. For more advanced use cases, introduction ofan explicit metaclass (possibly made available as a required base class) willstill be necessary in order to support Python 3.
In the body of an__autodecorate__ method, as in any other class method,__class__ will be bound to the class declaring the method, while thevalue passed in may be a subclass.
This makes it relatively straightforward to skip processing the base classif necessary:
classExample:def__autodecorate__(cls):cls=super().__autodecorate__()# Don't process the base classifclsis__class__:return# Process subclasses here...
As an implicit decorator,__autodecorate__ is able to relatively easilyreplace the defined class with a different kind of object. Technicallycustom metaclasses and even__new__ methods can already do thisimplicitly, but the decorator model makes such code much easier to understandand implement.
classBuildDict:def__autodecorate__(cls):cls=super().__autodecorate__()# Don't process the base classifclsis__class__:return# Convert subclasses to ordinary dictionariesreturncls.__dict__.copy()
It’s not clear why anyone would ever do this implicitly based on inheritancerather than just using an explicit decorator, but the possibility seems worthnoting.
namespace concept worth the extra complexity?Unlike the new__autodecorate__ hook the proposednamespace keywordargument is not automatically inherited by subclasses. Given the way thisproposal is currently written , the only way to get a special namespace usedconsistently in subclasses is still to write a custom metaclass with asuitable__prepare__ implementation.
Changing the custom namespace factory to also be inherited wouldsignificantly increase the complexity of this proposal, and introduce anumber of the same potential base class conflict issues as arise with theuse of custom metaclasses.
Eric Snow has put forward aseparate proposalto instead make the execution namespace for class bodies an ordered dictionaryby default, and capture the class attribute definition order for futurereference as an attribute (e.g.__definition_order__) on the class object.
Eric’s suggested approach may be a better choice for a new default behaviourfor type that combines well with the proposed__autodecorate__ hook,leaving the more complex configurable namespace factory idea to a custommetaclass like the one shown below.
The newnamespace keyword in the class header enables a number ofinteresting options for controlling the way a class is initialised,including some aspects of the object models of both Javascript and Ruby.
All of the examples below are actually possible today through the use of acustom metaclass:
classCustomNamespace(type):@classmethoddef__prepare__(meta,name,bases,*,namespace=None,**kwds):parent_namespace=super().__prepare__(name,bases,**kwds)returnnamespace()ifnamespaceisnotNoneelseparent_namespacedef__new__(meta,name,bases,ns,*,namespace=None,**kwds):returnsuper().__new__(meta,name,bases,ns,**kwds)def__init__(cls,name,bases,ns,*,namespace=None,**kwds):returnsuper().__init__(name,bases,ns,**kwds)
The advantage of implementing the new keyword directly intype.__prepare__ is that theonly persistent effect is thenthe change in the underlying storage of the class attributes. The metaclassof the class remains unchanged, eliminating many of the drawbackstypically associated with these kinds of customisations.
classOrderedClass(namespace=collections.OrderedDict):a=1b=2c=3
seed_data=dict(a=1,b=2,c=3)classPrepopulatedClass(namespace=seed_data.copy):pass
classNewClass(namespace=Prototype.__dict__.copy):pass
Note
Just because the PEP makes itpossible to do this relativelycleanly doesn’t mean anyoneshould do this!
fromcollectionsimportMutableMapping# The MutableMapping + dict combination should give something that# generally behaves correctly as a mapping, while still being accepted# as a class namespaceclassClassNamespace(MutableMapping,dict):def__init__(self,cls):self._cls=clsdef__len__(self):returnlen(dir(self._cls))def__iter__(self):forattrindir(self._cls):yieldattrdef__contains__(self,attr):returnhasattr(self._cls,attr)def__getitem__(self,attr):returngetattr(self._cls,attr)def__setitem__(self,attr,value):setattr(self._cls,attr,value)def__delitem__(self,attr):delattr(self._cls,attr)defextend(cls):returnlambda:ClassNamespace(cls)classExample:passclassExtendedExample(namespace=extend(Example)):a=1b=2c=3>>>Example.a,Example.b,Example.c(1,2,3)
__autodecorate__ fromtype.__init__Calling the new hook automatically fromtype.__init__, would achieve mostof the goals of this PEP. However, using that approach would mean that__autodecorate__ implementations would be unable to call any methods thatrelied on the__class__ reference (or used the zero-argument form ofsuper()), and could not make use of those features themselves.
The current design instead ensures that the implicit decorator hook is ableto do anything an explicit decorator can do by running it after the initialclass creation is already complete.
__init_class__Earlier versions of the PEP used the name__init_class__ for the nameof the new hook. There were three significant problems with this name:
__init_class__ or__class_init__type.__init__, which is not the caseThe new name__autodecorate__ was chosen to make it clear that the newinitialisation hook is most usefully thought of as an implicitly invokedclass decorator, rather than as being like an__init__ method.
__autodecorate__Originally, this PEP required the explicit use of@classmethod on the__autodecorate__ decorator. It was made implicit since there’s nosensible interpretation for leaving it out, and that case would need to bedetected anyway in order to give a useful error message.
This decision was reinforced after noticing that the user experience ofdefining__prepare__ and forgetting the@classmethod methoddecorator is singularly incomprehensible (particularly sincePEP 3115documents it as an ordinary method, and the current documentation doesn’texplicitly say anything one way or the other).
__autodecorate__ implicitly static, like__new__While it accepts the class to be instantiated as the first argument,__new__ is actually implicitly treated as a static method rather thanas a class method. This allows it to be readily extracted from itsdefining class and called directly on a subclass, rather than beingcoupled to the class object it is retrieved from.
Such behaviour initially appears to be potentially useful for thenew__autodecorate__ hook, as it would allow__autodecorate__methods to readily be used as explicit decorators on other classes.
However, that apparent support would be an illusion as it would only workcorrectly if invoked on a subclass, in which case the method can just asreadily be retrieved from the subclass and called that way. Unlike__new__, there’s no issue with potentially changing method signatures atdifferent points in the inheritance chain.
At one point, this PEP proposed that the class namespace be passeddirectly as a keyword argument, rather than passing a factory function.However, this encourages an unsupported behaviour (that is, passing thesame namespace to multiple classes, or retaining direct write accessto a mapping used as a class namespace), so the API was switched tothe factory function version.
A reference implementation for__autodecorate__ has been posted to theissue tracker. It uses the original__init_class__ naming. does not yetallow the implicit decorator to replace the class with a different object anddoes not implement the suggestednamespace parameter fortype.__prepare__.
This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0422.rst
Last modified:2025-02-01 08:59:27 GMT