Wouldn’t be cool to write a program that generates programs or code? The answer to that is closely related to the concept of metaprogramming and metaclasses (in Python).
In this post, you will learn how to write code that generates code and some of the situations that can make this technique very useful.
Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why).
Tim Peters
Introduction
A program usually takes an input and calculate an output.
Imagine now that instead of using the input to calculate the output, the program can modify itself to calculate the output?
Will this be useful?
Let’s analyze it according to the advantages.
Possible advantages:
- The code we must write to solve a specific problem will be less
- You can avoid code repetition. This can be one of the main problems in programming, especially for beginners.
- Better code and easier to maintain
Does it sound good?
In short, meta-programming is about programs that can write programs (or extend their codebase).
This technique is more than 40 years old. It was first used in Lisp. It was also used in Smalltalk in the ’80s.
You should notice that meta-programming is not the same as using dynamic programming languages to create a program.
Meta-programming
We have two main approaches for meta-programming:
- Generate programs
- Modify programs
Both approaches are valid under different situations, to solve different types of problems.
Program generation
Probably the best example of program generation is the programs that generate a parser.
A parser is the part of a compiler or interpreter that breaks a sentence (line of code) into words or units. These “units” are usually called tokens in compiler techniques.
Examples of programs generators:
- Bison: parser generator
- Flex: the fast lexical analyser
- Gperf: ” For a given list of strings, it produces a hash function and hash table, in form of C or C++ code, “
- CRUD generators for different frameworks. In this link, you will find several generators for different programming languages and frameworks.
Why are the previous tools implemented as a code generator instead of a function or class or module?
The main reason is that the input is complex and hard to represent in a programming language. You will need certain structures (like tables) to represent and implement operations.
Several aspects of the result we expect to get are adaptable. So, we can specify portions of code and generate the rest depending on the input.
Program modification
We usually create classes at design time. In other words, we first design the class, then we implement it in a specific programming language, then we execute our program.
If you are not familiar with Object-Oriented Programming, check this post and this one.
But what happens if we need to create a class at run time? or if we need to modify the behaviour of certain objects at run-time?
Usually, we define the behavior of an object by implementing methods within a class. Sometimes, the behaviour can change depending on different things.
In Object-Oriented Programming, we create objects using classes. So, we can understand a class as a template to create objects.
But, how classes are created?
What if we can personalize the way that a class is created? Can we create a template to create classes?
The simple answer is yes, we can. We do that by using metaclasses.
Metaclasses in Python
A metaclass is a class that have other classes as instances, instead of objects. So, a class can be an instance of a metaclass, while an object is an instance of a class.
By using metaclasses, we can:
- create classes
- personalize how a class is created
- enrich a class with functionalities like the ability to store logs, persistence, security features, among others.
- Specify the type of relations between classes.
- Modify the behavior of future objects when the class is created.
We can understand this as a way of aspect-oriented programming or as a generalization of the adapter pattern.
We can use this technique in several situations. Some of them are:
- We don’t know what classes we need as part of our solution at design time.
- To add behavior to classes. For instance, we want to add access methods to a class (getters and setters) for each attribute, without writing the boring code.
- Generate “boring code”. For instance, the CRUD forms for a web application.
In python, we can use the class type to create new types (classes).
type(name, bases, dct)
- name: the name of the new class
- bases: base classes
- dct: a dictionary with the methods
if __name__ == ‘__main__’:
Greeting = type(‘Greeting’, (), {‘hi’: lambda self: ‘Hi metaworld!’})
g = Greeting()
print(g.hi())
If you execute the code below, you will get the following output.
In Python, a class that inherits from the class type is a metaclass.
There are two main methods that we should implement when creating a metaclass:
- __new__: Crea una instancia de una clase
- __init__: Inicializa una instancia
- __call__: Allows an instance to be used as a function. When used in a metaclass, helps us to handle how the instances are created.
Example 1: Singleton pattern using metaclasses in Python
In several cases, we have to ensure that a certain class has only one instance. If we do not ensure that, it can create inconsistencies in our program.
For instance, imagine we are creating software to automate the operation of a certain CompanyX. One class from our design should be CompanyX.
We should ensure that we have only one instance of this class throughout our program. Otherwise, we will have two (different) objects representing only one concept: CompanyX.
Let’s see the code for our example. This code was made in Python3.
class Singleton(type):
def __init__(cls, name, bases, dct):
cls.__instance = None
type.__init__(cls, name, bases, dct)
def __call__(cls, *args, **kw):
if cls.__instance is None:
cls.__instance = type.__call__(cls, *args, **kw)
return cls.__instance
class CompanyX(metaclass=Singleton):
pass
if __name__ == '__main__':
company1 = CompanyX()
company2 = CompanyX()
print(company1 == company2)
If we execute the code above, the console will show True. If you remove the metaclass Singleton from the class CompanyX signature, you will have False as a result.
I highly recommend you test the two versions of the code (with and without the metaclass), so you can experience it yourself.
Example 2: Adding access methods (setters and getters) using metaclasses in Python
Another example of the use of metaclass can be adding “boring code” to our classes.
When we create a class, we usually need access methods. Programming languages like swift includes them by default, two (get and set) for each class attribute. C# also has a way of defining them without implementation.
Let’s see how we can use a metaclass to generate these access methods without having to write them every time we implement a new class.
class AccessMethods(type):
def __init__(self, name, bases, dic):
type.__init__(self, name, bases, dic)
if '_%s__get_set' % name in dic:
for attr in dic['_%s__get_set' % name]:
f = lambda s, a=attr: getattr(s, a)
setattr(self, 'get_' + attr, f)
f = lambda s, value, a=attr: setattr(s, a, value)
setattr(self, 'set_' + attr, f)
class Person(metaclass=AccessMethods):
__get_set = ['name', 'surname', 'age']
def __init__(self, name, surname, age):
self.name = name
self.surname = surname
self.age = age
In the line __get_set = [‘name’, ‘surname’, ‘age’], we just specify to the metaclass to create a get and set method for each specified attribute. Sometimes, we don’t need access methods for all attributes. In those cases, we can control what attribute have the access methods by adding or removing the attribute name from __get_set.
Notice the class Person does not have any access method. However, if you execute the code below it will just work.
if __name__ == '__main__':
p1 = Person('name', 'surname', 15)
p1.set_name('John')
print(p1.get_name())
Summary
Metaclasses can be very useful when used properly.
The main advantages are:
- We write less code
- You can avoid code repetition
- We can get better code
- Our code can be easier to maintain
H@ppy coding!