Movatterモバイル変換

You are notlogged in. Your edit will be placed in a queue until it ispeer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

Rev

Required fields *

What are metaclasses in Python?

What aremetaclasses? What are they used for?

Answer*

Classes are objects
=

Before delving into metaclasses, a solid grasp of Python classes is beneficial. Python holds a particularly distinctive concept of classes, a notion it adopts from the Smalltalk language.

In most languages, classes are descriptions of how to create an object. That is somewhat true in Python too:

 >>> class ObjectCreator(object):
 ... pass
 
 >>> my_object = ObjectCreator()
 >>> print(my_object)
 <__main__.ObjectCreator object at 0x8974f2c>

But classes are more than that in Python. **Classes are objects too.**

Yes, objects.

When a Python script runs, every line of code is executed from top to bottom. When the Python interpreter encounters the `class` keyword, Python creates an **object** out of the "description" of the class that follows. Thus, the following instruction

 >>> class ObjectCreator(object):
 ... pass

...creates an *object* with the name `ObjectCreator`!

This object (the class) is itself capable of creating objects (called *instances*).

But still, it's an object. Therefore, like all objects:

- you can assign it to a variable1

 ```
 JustAnotherVariable = ObjectCreator
 ```
- you can attach attributes to it

 ```
 ObjectCreator.class_attribute = 'foo'
 ```
- you can pass it as a function parameter

 ```
 print(ObjectCreator)
 ```

1 Note that merely assigning it to another variable doesn't change the class's `__name__`, i.e.,

```
>>> print(JustAnotherVariable)
 <class '__main__.ObjectCreator'>

>>> print(JustAnotherVariable())
 <__main__.ObjectCreator object at 0x8997b4c>
```



Creating classes dynamically
=

Since classes are objects, you can create them on the fly, like any object.

First, you can create a class in a function using `class`:

 >>> def choose_class(name):
 ... if name == 'foo':
 ... class Foo(object):
 ... pass
 ... return Foo # return the class, not an instance
 ... else:
 ... class Bar(object):
 ... pass
 ... return Bar
 
 >>> MyClass = choose_class('foo')
 
 >>> print(MyClass) # the function returns a class, not an instance
 <class '__main__.Foo'>
 
 >>> print(MyClass()) # you can create an object from this class
 <__main__.Foo object at 0x89c6d4c>

But it's not so dynamic, since you still have to write the whole class yourself.

Since classes are objects, they must be generated by something.

When you use the `class` keyword, Python creates this object automatically. But as
with most things in Python, it gives you a way to do it manually.

Remember the function `type`? The good old function that lets you know what
type an object is:

 >>> print(type(1))
 <class 'int'>
 
 >>> print(type("1"))
 <class 'str'>
 
 >>> print(type(ObjectCreator))
 <class 'type'>
 
 >>> print(type(ObjectCreator()))
 <class '__main__.ObjectCreator'>

Well, [`type`][1] also has a completely different ability: it can create classes on the fly. `type` can take the description of a class as parameters,
and return a class.

(I know, it's silly that the same function can have two completely different uses according to the parameters you pass to it. It's an issue due to backward
compatibility in Python)

`type` works this way:

 type(name, bases, attrs)

Where:

- **`name`**: name of the class
- **`bases`**: tuple of the parent class (for inheritance, can be empty)
- **`attrs`**: dictionary containing attributes names and values

e.g.:

 >>> class MyShinyClass(object):
 ... pass

can be created manually this way:

 >>> MyShinyClass = type('MyShinyClass', (), {}) # returns a class object
 >>> print(MyShinyClass)
 <class '__main__.MyShinyClass'>
 
 >>> print(MyShinyClass()) # create an instance with the class
 <__main__.MyShinyClass object at 0x8997cec>

You'll notice that we use `MyShinyClass` as the name of the class
and as the variable to hold the class reference. They can be different,
but there is no reason to complicate things.

`type` accepts a dictionary to define the attributes of the class. So:

 >>> class Foo(object):
 ... bar = True

Can be translated to:

 >>> Foo = type('Foo', (), {'bar':True})

And used as a normal class:

 >>> print(Foo)
 <class '__main__.Foo'>
 
 >>> print(Foo.bar)
 True
 
 >>> f = Foo()
 >>> print(f)
 <__main__.Foo object at 0x8a9b84c>
 
 >>> print(f.bar)
 True

And of course, you can inherit from it, so:

 >>> class FooChild(Foo):
 ... pass

would be:

 >>> FooChild = type('FooChild', (Foo,), {})
 >>> print(FooChild)
 <class '__main__.FooChild'>
 
 >>> print(FooChild.bar) # bar is inherited from Foo
 True

Eventually, you'll want to add methods to your class. Just define a function
with the proper signature and assign it as an attribute.

 >>> def echo_bar(self):
 ... print(self.bar)
 
 >>> FooChild = type('FooChild', (Foo,), {'echo_bar': echo_bar})
 
 >>> hasattr(Foo, 'echo_bar')
 False
 
 >>> hasattr(FooChild, 'echo_bar')
 True
 
 >>> my_foo = FooChild()
 >>> my_foo.echo_bar()
 True

And you can add even more methods after you dynamically create the class, just like adding methods to a normally created class object.

 >>> def echo_bar_more(self):
 ... print('yet another method')
 
 >>> FooChild.echo_bar_more = echo_bar_more
 >>> hasattr(FooChild, 'echo_bar_more')
 True

You see where we are going: in Python, classes are objects, and you can create a class on the fly, dynamically.

This is what Python does when you use the keyword `class`, and it does so by using a metaclass.

What are metaclasses (finally)
=

Metaclasses are the 'stuff' that creates classes.

You define classes in order to create objects, right?

But we learned that Python classes are objects.

Well, metaclasses are what create these objects. They are the classes' classes,
you can picture them this way:

 MyClass = MetaClass()
 my_object = MyClass()

You've seen that `type` lets you do something like this:

 MyClass = type('MyClass', (), {})

It's because the function `type` is in fact a metaclass. `type` is the
metaclass Python uses to create all classes behind the scenes.

Now you wonder "why the heck is it written in lowercase, and not `Type`?"

Well, I guess it's a matter of consistency with `str`, the class that creates
strings objects, and `int` the class that creates integer objects. `type` is
just the class that creates class objects.

You see that by checking the `__class__` attribute.

Everything, and I mean everything, is an object in Python. That includes integers,
strings, functions and classes. All of them are objects. And all of them have
been created from a class:

 >>> age = 35
 >>> age.__class__
 <type 'int'>
 
 >>> name = 'bob'
 >>> name.__class__
 <type 'str'>
 
 >>> def foo(): pass
 >>> foo.__class__
 <type 'function'>
 
 >>> class Bar(object): pass
 >>> b = Bar()
 >>> b.__class__
 <class '__main__.Bar'>

Now, what is the `__class__` of any `__class__` ?

 >>> age.__class__.__class__
 <type 'type'>
 
 >>> name.__class__.__class__
 <type 'type'>
 
 >>> foo.__class__.__class__
 <type 'type'>
 
 >>> b.__class__.__class__
 <type 'type'>

So, a metaclass is just the stuff that creates class objects.

You can call it a 'class factory' if you wish.

`type` is the built-in metaclass Python uses, but of course, you can create your
own metaclass.

The [`__metaclass__`][2] attribute
=

In Python 2, you can add a `__metaclass__` attribute when you write a class (see next section for the Python 3 syntax):

 class Foo(object):
 __metaclass__ = something...
 [...]

If you do so, Python will use the metaclass to create the class `Foo`.

Careful, it's tricky.

You write `class Foo(object)` first, but the class object `Foo` is not created
in memory yet.

Python will look for `__metaclass__` in the class definition. If it finds it,
it will use it to create the class object `Foo`. If it doesn't, it will use
`type` to create the class.

Read that several times.

When you do:

 class Foo(Bar):
 pass

Python does the following:

Is there a `__metaclass__` attribute in `Foo`?

If yes, create in-memory a class object (I said a class object, stay with me here), with the name `Foo` by using what is in `__metaclass__`.

If Python can't find `__metaclass__`, it will look for a `__metaclass__` at the MODULE level, and try to do the same (but only for classes that don't inherit anything, basically old-style classes).

Then if it can't find any `__metaclass__` at all, it will use the `Bar`'s (the first parent) own metaclass (which might be the default `type`) to create the class object.

Be careful here that the `__metaclass__` attribute will not be inherited, the metaclass of the parent (`Bar.__class__`) will be. If `Bar` used a `__metaclass__` attribute that created `Bar` with `type()` (and not `type.__new__()`), the subclasses will not inherit that behavior.

Now the big question is, what can you put in `__metaclass__`?

The answer is something that can create a class.

And what can create a class? `type`, or anything that subclasses or uses it.

Metaclasses in Python 3
=

The syntax to set the metaclass has been changed in Python 3:

 class Foo(object, metaclass=something):
 ...

i.e. the `__metaclass__` attribute is no longer used, in favor of a keyword argument in the list of base classes.

The behavior of metaclasses however stays [largely the same](https://www.python.org/dev/peps/pep-3115/).

One thing added to metaclasses in Python 3 is that you can also pass attributes as keyword-arguments into a metaclass, like so:

 class Foo(object, metaclass=something, kwarg1=value1, kwarg2=value2):
 ...

Read the section below for how Python handles this.

Custom metaclasses
=

The main purpose of a metaclass is to change the class automatically,
when it's created.

You usually do this for APIs, where you want to create classes matching the
current context.

Imagine a stupid example, where you decide that all classes in your module
should have their attributes written in uppercase. There are several ways to
do this, but one way is to set `__metaclass__` at the module level.

This way, all classes of this module will be created using this metaclass,
and we just have to tell the metaclass to turn all attributes to uppercase.

Luckily, `__metaclass__` can actually be any callable, it doesn't need to be a
formal class (I know, something with 'class' in its name doesn't need to be
a class, go figure... but it's helpful).

So we will start with a simple example, by using a function.

 # the metaclass will automatically get passed the same argument
 # that you usually pass to `type`
 def upper_attr(future_class_name, future_class_parents, future_class_attrs):
 """
 Return a class object, with the list of its attribute turned
 into uppercase.
 """
 # pick up any attribute that doesn't start with '__' and uppercase it
 uppercase_attrs = {
 attr if attr.startswith("__") else attr.upper(): v
 for attr, v in future_class_attrs.items()
 }
 
 # let `type` do the class creation
 return type(future_class_name, future_class_parents, uppercase_attrs)
 
 __metaclass__ = upper_attr # this will affect all classes in the module
 
 class Foo(): # global __metaclass__ won't work with "object" though
 # but we can define __metaclass__ here instead to affect only this class
 # and this will work with "object" children
 bar = 'bip'

Let's check:

 >>> hasattr(Foo, 'bar')
 False
 
 >>> hasattr(Foo, 'BAR')
 True
 
 >>> Foo.BAR
 'bip'

Now, let's do exactly the same, but using a real class for a metaclass:

 # remember that `type` is actually a class like `str` and `int`
 # so you can inherit from it
 class UpperAttrMetaclass(type):
 # __new__ is the method called before __init__
 # it's the method that creates the object and returns it
 # while __init__ just initializes the object passed as parameter
 # you rarely use __new__, except when you want to control how the object
 # is created.
 # here the created object is the class, and we want to customize it
 # so we override __new__
 # you can do some stuff in __init__ too if you wish
 # some advanced use involves overriding __call__ as well, but we won't
 # see this
 def __new__(
 upperattr_metaclass,
 future_class_name,
 future_class_parents,
 future_class_attrs
 ):
 uppercase_attrs = {
 attr if attr.startswith("__") else attr.upper(): v
 for attr, v in future_class_attrs.items()
 }
 return type(future_class_name, future_class_parents, uppercase_attrs)

Let's rewrite the above, but with shorter and more realistic variable names now that we know what they mean:

 class UpperAttrMetaclass(type):
 def __new__(cls, clsname, bases, attrs):
 uppercase_attrs = {
 attr if attr.startswith("__") else attr.upper(): v
 for attr, v in attrs.items()
 }
 return type(clsname, bases, uppercase_attrs)

You may have noticed the extra argument `cls`. There is
nothing special about it: `__new__` always receives the class it's defined in, as the first parameter. Just like you have `self` for ordinary methods which receive the instance as the first parameter, or the defining class for class methods.

But this is not proper OOP. We are calling `type` directly and we aren't overriding or calling the parent's `__new__`. Let's do that instead:

 class UpperAttrMetaclass(type):
 def __new__(cls, clsname, bases, attrs):
 uppercase_attrs = {
 attr if attr.startswith("__") else attr.upper(): v
 for attr, v in attrs.items()
 }
 return type.__new__(cls, clsname, bases, uppercase_attrs)

We can make it even cleaner by using `super`, which will ease inheritance (because yes, you can have metaclasses, inheriting from metaclasses, inheriting from type):

 class UpperAttrMetaclass(type):
 def __new__(cls, clsname, bases, attrs):
 uppercase_attrs = {
 attr if attr.startswith("__") else attr.upper(): v
 for attr, v in attrs.items()
 }
 
 # Python 2 requires passing arguments to super:
 return super(UpperAttrMetaclass, cls).__new__(
 cls, clsname, bases, uppercase_attrs)
 
 # Python 3 can use no-arg super() which infers them:
 return super().__new__(cls, clsname, bases, uppercase_attrs)

Oh, and in Python 3 if you do this call with keyword arguments, like this:

 class Foo(object, metaclass=MyMetaclass, kwarg1=value1):
 ...

It translates to this in the metaclass to use it:

 class MyMetaclass(type):
 def __new__(cls, clsname, bases, dct, kwarg1=default):
 ...

That's it. There is really nothing more about metaclasses.

The reason behind the complexity of the code using metaclasses is not because
of metaclasses, it's because you usually use metaclasses to do twisted stuff
relying on introspection, manipulating inheritance, vars such as `__dict__`, etc.

Indeed, metaclasses are especially useful to do black magic, and therefore
complicated stuff. But by themselves, they are simple:

- intercept a class creation
- modify the class
- return the modified class

Why would you use metaclasses classes instead of functions?
=

Since `__metaclass__` can accept any callable, why would you use a class
since it's obviously more complicated?

There are several reasons to do so:

- The intention is clear. When you read `UpperAttrMetaclass(type)`, you know
 what's going to follow
- You can use OOP. Metaclass can inherit from metaclass, override parent methods. Metaclasses can even use metaclasses.
- Subclasses of a class will be instances of its metaclass if you specified a metaclass-class, but not with a metaclass-function.
- You can structure your code better. You never use metaclasses for something as trivial as the above example. It's usually for something complicated. Having the ability to make several methods and group them in one class is very useful to make the code easier to read.
- You can hook on `__new__`, `__init__` and `__call__`. Which will allow you to do different stuff, Even if usually you can do it all in `__new__`,
 some people are just more comfortable using `__init__`.
- These are called metaclasses, damn it! It must mean something!

Why would you use metaclasses?
=

Now the big question. Why would you use some obscure error-prone feature?

Well, usually you don't:

> Metaclasses are deeper magic that
> 99% of users should never worry about it.
> If you wonder whether you need them,
> you don't (the people who actually
> need them know with certainty that
> they need them, and don't need an
> explanation about why).

*Python Guru Tim Peters*

The main use case for a metaclass is creating an API. A typical example of this is the Django ORM. It allows you to define something like this:

 class Person(models.Model):
 name = models.CharField(max_length=30)
 age = models.IntegerField()

But if you do this:

 person = Person(name='bob', age='35')
 print(person.age)

It won't return an `IntegerField` object. It will return an `int`, and can even take it directly from the database.

This is possible because `models.Model` defines `__metaclass__` and
it uses some magic that will turn the `Person` you just defined with simple statements
into a complex hook to a database field.

Django makes something complex look simple by exposing a simple API
and using metaclasses, recreating code from this API to do the real job
behind the scenes.

The last word
=

First, you know that classes are objects that can create instances.

Well, in fact, classes are themselves instances. Of metaclasses.

 >>> class Foo(object): pass
 >>> id(Foo)
 142630324

Everything is an object in Python, and they are all either instance of classes
or instances of metaclasses.

Except for `type`.

`type` is actually its own metaclass. This is not something you could
reproduce in pure Python, and is done by cheating a little bit at the implementation
level.

Secondly, metaclasses are complicated. You may not want to use them for
very simple class alterations. You can change classes by using two different techniques:

- [monkey patching](http://en.wikipedia.org/wiki/Monkey_patch)
- class decorators

99% of the time you need class alteration, you are better off using these.

But 98% of the time, you don't need class alteration at all.

[1]: http://docs.python.org/2/library/functions.html#type
[2]: http://docs.python.org/2/reference/datamodel.html?highlight=\__metaclass_\_#\__metaclass_\_

tips on writing great answers.

Draft saved

Draft discarded

Edit Summary*

Cancel

80
It appears that in Djangomodels.Model it does not use__metaclass__ but ratherclass Model(metaclass=ModelBase): to reference aModelBase class which then does the aforementioned metaclass magic. Great post! Here's the Django source:github.com/django/django/blob/master/django/db/models/…
Max Goodridge
– Max Goodridge
2017-04-12 13:18:59 +00:00
CommentedApr 12, 2017 at 13:18
40
<<Be careful here that the__metaclass__ attribute will not be inherited, the metaclass of the parent (Bar.__class__) will be. IfBar used a__metaclass__ attribute that createdBar withtype() (and nottype.__new__()), the subclasses will not inherit that behavior.>> -- Could you/someone please explain a bit deeper this passage?
petrux
– petrux
2017-04-25 21:32:44 +00:00
CommentedApr 25, 2017 at 21:32
37
@MaxGoodridge That's the Python 3 syntax for metaclasses. SeePython 3.6 Data model VSPython 2.7 Data model
TBBle
– TBBle
2017-06-13 13:22:27 +00:00
CommentedJun 13, 2017 at 13:22
32
Now you wonder why the heck is it written in lowercase, and not Type? - well because it's implemented in C - it's the same reason defaultdict is lowercase while OrderedDict (in python 2) is normal CamelCase
Mr_and_Mrs_D
– Mr_and_Mrs_D
2017-09-29 10:47:27 +00:00
CommentedSep 29, 2017 at 10:47
35
It's a community wiki answer (so, those who commented with corrections/improvements might consider editing their comments into the answer, if they're sure they are correct).
Brōtsyorfuzthrāx
– Brōtsyorfuzthrāx
2017-11-08 08:59:31 +00:00
CommentedNov 8, 2017 at 8:59

| Show21 more comments

How to Edit

Correct minor typos or mistakes
Clarify meaning without changing it
Add related resources or links
Always respect the author’s intent
Don’t use edits to reply to the author

How to Format

create code fences with backticks ` or tildes ~
```
like so
```
add language identifier to highlight code
```python
def function(foo):
print(foo)
```
put returns between paragraphs
for linebreak add 2 spaces at end
_italic_ or**bold**
indent code by 4 spaces
backtick escapes`like _so_`
quote by placing > at start of line
to make links (use https whenever possible)
<https://example.com>
[example](https://example.com)
<a href="https://example.com">example</a>

formatting help »
answering help »

How to Tag

A tag is a keyword or label that categorizes your question with other, similar questions. Choose one or more (up to 5) tags that will help answerers to find and interpret your question.

complete the sentence:my question is about...
use tags that describe things or concepts that areessential, not incidental to your question
favor usingexisting popular tags
read the descriptions that appear below the tag

If your question is primarily about a topic for which you can't find a tag:

combine multiple words into single-words with hyphens (e.g.python-3.x), up to a maximum of 35 characters
creating new tags is a privilege; if you can't yet create a tag you need, then post this question without it, thenask the community to create it for you

popular tags »

lang-py

Movatterモバイル変換

Collectives™ on Stack Overflow

What are metaclasses in Python?

Answer*