Abstract

This PEP proposes changing the syntax for declaring metaclasses,and alters the semantics for how classes with metaclasses areconstructed.

Rationale

There are two rationales for this PEP, both of which are somewhatsubtle.

The primary reason for changing the way metaclasses work, is thatthere are a number of interesting use cases that require themetaclass to get involved earlier in the class construction processthan is currently possible. Currently, the metaclass mechanism isessentially a post-processing step. With the advent of classdecorators, much of these post-processing chores can be taken overby the decorator mechanism.

In particular, there is an important body of use cases where itwould be useful to preserve the order in which a class members aredeclared. Ordinary Python objects store their members in adictionary, in which ordering is unimportant, and members areaccessed strictly by name. However, Python is often used tointerface with external systems in which the members are organizedaccording to an implicit ordering. Examples include declaration of Cstructs; COM objects; Automatic translation of Python classes intoIDL or database schemas, such as used in an ORM; and so on.

In such cases, it would be useful for a Python programmer to specifysuch ordering directly using the declaration order of class members.Currently, such orderings must be specified explicitly, using someother mechanism (see the ctypes module for an example.)

Unfortunately, the current method for declaring a metaclass doesnot allow for this, since the ordering information has already beenlost by the time the metaclass comes into play. By allowing themetaclass to get involved in the class construction process earlier,the new system allows the ordering or other early artifacts ofconstruction to be preserved and examined.

There proposed metaclass mechanism also supports a number of otherinteresting use cases beyond preserving the ordering of declarations.One use case is to insert symbols into the namespace of the classbody which are only valid during class construction. An example ofthis might be “field constructors”, small functions that are used inthe creation of class members. Another interesting possibility issupporting forward references, i.e. references to Pythonsymbols that are declared further down in the class body.

The other, weaker, rationale is purely cosmetic: The current methodfor specifying a metaclass is by assignment to the special variable__metaclass__, which is considered by some to be aesthetically lessthan ideal. Others disagree strongly with that opinion. This PEPwill not address this issue, other than to note it, since aestheticdebates cannot be resolved via logical proofs.

Specification

In the new model, the syntax for specifying a metaclass is via akeyword argument in the list of base classes:

classFoo(base1,base2,metaclass=mymeta):...

Additional keywords will also be allowed here, and will be passed tothe metaclass, as in the following example:

classFoo(base1,base2,metaclass=mymeta,private=True):...

Note that this PEP makes no attempt to define what these otherkeywords might be - that is up to metaclass implementors todetermine.

More generally, the parameter list passed to a class definition willnow support all of the features of a function call, meaning that youcan now use*args and**kwargs-style arguments in the class baselist:

classFoo(*bases,**kwds):...

Invoking the Metaclass

In the current metaclass system, the metaclass object can be anycallable type. This does not change, however in order to fullyexploit all of the new features the metaclass will need to have anextra attribute which is used during class pre-construction.

This attribute is named__prepare__, which is invoked as a functionbefore the evaluation of the class body. The__prepare__ functiontakes two positional arguments, and an arbitrary number of keywordarguments. The two positional arguments are:

The interpreter always tests for the existence of__prepare__ beforecalling it; If it is not present, then a regular dictionary is used,as illustrated in the following Python snippet.

defprepare_class(name,*bases,metaclass=None,**kwargs):ifmetaclassisNone:metaclass=compute_default_metaclass(bases)prepare=getattr(metaclass,'__prepare__',None)ifprepareisnotNone:returnprepare(name,bases,**kwargs)else:returndict()

The example above illustrates how the arguments to ‘class’ areinterpreted. The class name is the first argument, followed byan arbitrary length list of base classes. After the base classes,there may be one or more keyword arguments, one of which can bemetaclass. Note that themetaclass argument is not includedinkwargs, since it is filtered out by the normal parameterassignment algorithm. (Note also thatmetaclass is akeyword-only argument as perPEP 3102.)

Even though__prepare__ is not required, the default metaclass(‘type’) implements it, for the convenience of subclasses callingit via super().

__prepare__ returns a dictionary-like object which is used to storethe class member definitions during evaluation of the class body.In other words, the class body is evaluated as a function block(just like it is now), except that the local variables dictionaryis replaced by the dictionary returned from__prepare__. Thisdictionary object can be a regular dictionary or a custom mappingtype.

This dictionary-like object is not required to support the fulldictionary interface. A dictionary which supports a limited set ofdictionary operations will restrict what kinds of actions can occurduring evaluation of the class body. A minimal implementation mightonly support adding and retrieving values from the dictionary - mostclass bodies will do no more than that during evaluation. For someclasses, it may be desirable to support deletion as well. Manymetaclasses will need to make a copy of this dictionary afterwards,so iteration or other means for reading out the dictionary contentsmay also be useful.

The__prepare__ method will most often be implemented as a classmethod rather than an instance method because it is called beforethe metaclass instance (i.e. the class itself) is created.

Once the class body has finished evaluating, the metaclass will becalled (as a callable) with the class dictionary, which is nodifferent from the current metaclass mechanism.

Typically, a metaclass will create a custom dictionary - either asubclass of dict, or a wrapper around it - that will containadditional properties that are set either before or during theevaluation of the class body. Then in the second phase, themetaclass can use these additional properties to further customizethe class.

An example would be a metaclass that uses information about theordering of member declarations to create a C struct. The metaclasswould provide a custom dictionary that simply keeps a record of theorder of insertions. This does not need to be a full ‘ordered dict’implementation, but rather just a Python list of (key,value) pairsthat is appended to for each insertion.

Note that in such a case, the metaclass would be required to dealwith the possibility of duplicate keys, but in most cases that istrivial. The metaclass can use the first declaration, the last,combine them in some fashion, or simply throw an exception. It’s upto the metaclass to decide how it wants to handle that case.

Example

Here’s a simple example of a metaclass which creates a list ofthe names of all class members, in the order that they weredeclared:

# The custom dictionaryclassmember_table(dict):def__init__(self):self.member_names=[]def__setitem__(self,key,value):# if the key is not already defined, add to the# list of keys.ifkeynotinself:self.member_names.append(key)# Call superclassdict.__setitem__(self,key,value)# The metaclassclassOrderedClass(type):# The prepare function@classmethoddef__prepare__(metacls,name,bases):# No keywords in this casereturnmember_table()# The metaclass invocationdef__new__(cls,name,bases,classdict):# Note that we replace the classdict with a regular# dict before passing it to the superclass, so that we# don't continue to record member names after the class# has been created.result=type.__new__(cls,name,bases,dict(classdict))result.member_names=classdict.member_namesreturnresultclassMyClass(metaclass=OrderedClass):# method1 goes in array element 0defmethod1(self):pass# method2 goes in array element 1defmethod2(self):pass

Sample Implementation

Guido van Rossum has created a patch which implements the newfunctionality:https://bugs.python.org/issue1681101

Alternate Proposals

Josiah Carlson proposed using the name ‘type’ instead of‘metaclass’, on the theory that what is really being specified isthe type of the type. While this is technically correct, it is alsoconfusing from the point of view of a programmer creating a newclass. From the application programmer’s point of view, the ‘type’that they are interested in is the class that they are writing; thetype of that type is the metaclass.

There were some objections in the discussion to the ‘two-phase’creation process, where the metaclass is invoked twice, once tocreate the class dictionary and once to ‘finish’ the class. Somepeople felt that these two phases should be completely separate, inthat there ought to be separate syntax for specifying the customdict as for specifying the metaclass. However, in most cases, thetwo will be intimately tied together, and the metaclass will mostlikely have an intimate knowledge of the internal details of theclass dict. Requiring the programmer to insure that the correct dicttype and the correct metaclass type are used together creates anadditional and unneeded burden on the programmer.

Another good suggestion was to simply use an ordered dict for allclasses, and skip the whole ‘custom dict’ mechanism. This was basedon the observation that most use cases for a custom dict were forthe purposes of preserving order information. However, this idea hasseveral drawbacks, first because it means that an ordered dictimplementation would have to be added to the set of built-in typesin Python, and second because it would impose a slight speed (andcomplexity) penalty on all class declarations. Later, several peoplecame up with ideas for use cases for custom dictionaries otherthan preserving field orderings, so this idea was dropped.

Backwards Compatibility

It would be possible to leave the existing__metaclass__ syntax inplace. Alternatively, it would not be too difficult to modify thesyntax rules of the Py3K translation tool to convert from the old tothe new syntax.

References

[1] [Python-3000] Metaclasses in Py3K (original proposal)https://mail.python.org/pipermail/python-3000/2006-December/005030.html

[2] [Python-3000] Metaclasses in Py3K (Guido’s suggested syntax)https://mail.python.org/pipermail/python-3000/2006-December/005033.html

[3] [Python-3000] Metaclasses in Py3K (Objections to two-phase init)https://mail.python.org/pipermail/python-3000/2006-December/005108.html

[4] [Python-3000] Metaclasses in Py3K (Always use an ordered dict)https://mail.python.org/pipermail/python-3000/2006-December/005118.html

Copyright

This document has been placed in the public domain.