Python Meta Classes CowBoy

Python metaclasses explained

Python is having obscure wonderful mechanism of classes and it’s implementation.   In python every thing is an object. If you define a class, that class it self is an object in memory and it is an instance of other class. You may call it as a class object. If you instantiate it you will get brand new object called instance( instance of class).

Probably, meta classes  are  sort of confusing concept. Many people are afraid of this concept, but believe me it is very nice and simple if you understand it.

In simple words, meta classes are classes which are responsible to create  a class object( in memory).

As mentioned above, when you define a class. A class object will be created in memory but behind the senses it is an instance of other class by default type. So, classes are created by meta classes. You can specify your custom meta class which is responsible to create a class.

Mostly, meta classes are used when you write APIs or frameworks. Like django uses metaclasses in models.

Meta Classes
Meta Classes

I would like to explain this concept by taking real world example, to get you good understanding. Let’s take a look at following picture.

Python MetaClasses Illustrated
Python MetaClasses Illustrated

As show in picture, A factory can be treated a metaclass. Where, it produces vending machine, can be considered as a class. Where, the class (vending machine) produces instances, which are cans, bottles etc.

An example representing normal class definition and instance creation

Python is having builtin function called isinstance. Using this function we can determine if any object is an instance of specified object.

In the above example we haven’t specified the metaclass. So, python would use default one. i.e type.

In python 2, we specify metaclass using magic method __metaclass__.
__metaclass__ can be specified either as a class attribute or global variable. When you specify it globally in module level, all classes will become instances of this metaclass. If you specify this as a class attribute, only that class will become instance of specified metaclass. In python3 this __meatclass__ attribute is removed in favour of using metaclass argument that you have to specify in class definition.

Implementation of simple class with metaclass in place

Here in this example, you can see class Foo is the instance of metaclass MetaClassThus, Foo gets created as an instance of MetaClass. And, i is an instance of class Foo. As you can see, we confirmed this relation using isinstance function.

Python metaclasses - instance, class and metaclass
Python metaclasses – instance, class and metaclass

There is no restriction over how many times one class can be used as a metaclass for other classes.

In the above example,  the meta class MetaCalss is used in both Foo and Bar classes. Like wise you can use it in as many classes you want. If you want to apply it for all module level classes. You better use global __metaclass__ attribute. As you can see in the following example

OutPut:

This module level __metaclass__ magic variable doesn’t work on new style classes as show in above code.

 

Python Metaclasses Key Notes

  • Metaclasses are callables
  • Subclasses inherit the metaclass
  • Restriction over multiple metaclasses in multiple inheritance
MetaClasses are callable

The MetaClass always need not be a class. You can use any callable as a metaclass. A Simple example demonstrates using callable (function) as a metaclass

If you are using callable as a metaclass class. It should have same signature(arguments) as type. That is,

Following example is useless but explanatory. You can hack the class creation using metaclass. You can see below example, Foo became 3 instead of class object as we did return 3 from metaclass.

Subclasses inherit the metaclass

Like all other attributes and methods subclasses inherit metaclass.

OutPut:

#TODO

Restriction over multiple metaclasses in inheritance

Classes can have multiple base classes. Those maetaclasses may have different metaclass. If so, every thing should be linear in the inheritance. That is, multi-level inheritance works fine. But, python is having restriction over having multiple metaclasses in multiple inheritance  from different bases. Following example shows that, raised exception because of having multiple metaclasses from two bases.

Where metaclasses used ?

The main use case for a metaclass is creating an API, best example of this is Django ORM.

If you are aware of django. We usually define a model as follows

But,

It won’t return an IntegerField object. Instead it will return int.

This hack is actually possible by defining  __metaclass__ on models.Model and it uses some magic that will turn the model Person you defined with simple statements into a complex hook to a database field.

Django makes complex look simple by exposing a simple API and using metaclasses inside.

Metaclasses in python3

In python3 we specify metaclass by passing keyword argument metaclass to class definition. This is one of the difference between python2 and python3.

Py3K translation tool can be used to convert old syntax to new syntax.

Summary

In python classes are actually instances of other classes (soft of)  called metaclasses. You can specify metaclass using magic method __metaclass__ either in class definition or gobally module level. These metaclasses are useful when you want customize class creation.

 

Conclusion

Metaclasses is the obscure feature in python. Use it, when you really need them.

Metaclasses are deeper magic that 99% of users should never worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why).

Python Guru Tim Peters

Best real life example where metaclasses are being used is Django ORM. As like Django if you are writing framework or API, you can leverage metaclasses and you really need them.

Please share. Let me know your comments and queries 😉 😉

References:

[1] https://www.python.org/dev/peps/pep-3115/
[2]http://stackoverflow.com/questions/2149846/customary-to-inherit-metaclasses-from-type
[3]http://stackoverflow.com/questions/100003/what-is-a-metaclass-in-python

2 thoughts on “Python metaclasses explained”

  1. Cool article!
    I remember some description of metaclasses from one of the first python books I read, but had mainly forgotten the description. It’s interesting to see it brought out in detailed examples. I had wondered how the Django ORM pulled off that magic. 🙂

    Thanks!
    jima

    1. I’m glad you liked this article. Django uses meta classes with models to convert model definition into nice API.

Leave a Reply

Your email address will not be published. Required fields are marked *