6.8. Prefer Class Decorators Over Metaclasses for Composable Class Extensions¶
Although metaclasses allow you to customize class creation in multiple ways (see Item 48: “Validate Subclasses with __init_subclass__” and Item 49: “Register Class Existence with __init_subclass__”), they still fall short of handling every situation that may arise.
For example, say that I want to decorate all of the methods of a class with a helper that prints arguments, return values, and exceptions raised. Here, I define the debugging decorator (see Item 26: “Define Function Decorators with functools.wraps” for background):
>>> from functools import wraps
>>>
>>> def trace_func(func):
>>> if hasattr(func, 'tracing'): # Only decorate once
>>> return func
>>>
>>> @wraps(func)
>>> def wrapper(*args, **kwargs):
>>> result = None
>>> try:
>>> result = func(*args, **kwargs)
>>> return result
>>> except Exception as e:
>>> result = e
>>> raise
>>>
>>> finally:
>>> print(f'{func.__name__}({args!r}, {kwargs!r}) -> ' f'{result!r}')
>>>
>>> wrapper.tracing = True
>>> return wrapper
I can apply this decorator to various special methods in my new dict subclass (see Item 43: “Inherit from collections.abc for Custom Container Types” for background):
>>> class TraceDict(dict):
>>> @trace_func
>>> def __init__(self, *args, **kwargs):
>>> super().__init__(*args, **kwargs)
>>>
>>> @trace_func
>>> def __setitem__(self, *args, **kwargs):
>>> return super().__setitem__(*args, **kwargs)
>>>
>>> @trace_func
>>> def __getitem__(self, *args, **kwargs):
>>> return super().__getitem__(*args, **kwargs)
>>>
>>> ...
And I can verify that these methods are decorated by interacting with an instance of the class:
>>> trace_dict = TraceDict([('hi', 1)])
>>> trace_dict['there'] = 2
>>> trace_dict['hi']
>>> try:
>>> trace_dict['does not exist']
>>> except KeyError:
>>> pass # Expected
__init__(({'hi': 1}, [('hi', 1)]), {}) -> None
__setitem__(({'hi': 1, 'there': 2}, 'there', 2), {}) -> None
__getitem__(({'hi': 1, 'there': 2}, 'hi'), {}) -> 1
__getitem__(({'hi': 1, 'there': 2}, 'does not exist'), {}) -> KeyError('does not exist')
The problem with this code is that I had to redefine all of the methods that I wanted to decorate with @trace_func. This is redundant boilerplate that’s hard to read and error prone. Further, if a new method is later added to the dict superclass, it won’t be decorated unless I also define it in TraceDict.
One way to solve this problem is to use a metaclass to automatically decorate all methods of a class. Here, I implement this behavior by wrapping each function or method in the new type with the trace_func decorator:
>>> import types
>>>
>>> trace_types = (
>>> types.MethodType,
>>> types.FunctionType,
>>> types.BuiltinFunctionType,
>>> types.BuiltinMethodType,
>>> types.MethodDescriptorType,
>>> types.ClassMethodDescriptorType)
>>>
>>> class TraceMeta(type):
>>> def __new__(meta, name, bases, class_dict):
>>> klass = super().__new__(meta, name, bases, class_dict)
>>>
>>> for key in dir(klass):
>>> value = getattr(klass, key)
>>> if isinstance(value, trace_types):
>>> wrapped = trace_func(value)
>>> setattr(klass, key, wrapped)
>>>
>>> return klass
Now, I can declare my dict subclass by using the TraceMeta metaclass and verify that it works as expected:
>>> class TraceDict(dict, metaclass=TraceMeta):
>>> pass
>>>
>>> trace_dict = TraceDict([('hi', 1)])
>>> trace_dict['there'] = 2
>>> trace_dict['hi']
>>> try:
>>> trace_dict['does not exist']
>>> except KeyError:
>>> pass # Expected
__new__((<class '__main__.TraceDict'>, [('hi', 1)]), {}) -> {}
__getitem__(({'hi': 1, 'there': 2}, 'hi'), {}) -> 1
__getitem__(({'hi': 1, 'there': 2}, 'does not exist'), {}) -> KeyError('does not exist')
This works, and it even prints out a call to __new__ that was missing from my earlier implementation. What happens if I try to use TraceMeta when a superclass already has specified a metaclass?
- class OtherMeta(type):
pass
- class SimpleDict(dict, metaclass=OtherMeta):
pass
- class TraceDict(SimpleDict, metaclass=TraceMeta):
pass
>>>
Traceback ...
TypeError: metaclass conflict: the metaclass of a derived
➥class must be a (non-strict) subclass of the metaclasses
➥of all its bases
This fails because TraceMeta does not inherit from OtherMeta. In theory, I can use metaclass inheritance to solve this problem by having OtherMeta inherit from TraceMeta:
>>> class TraceMeta(type):
>>> ...
>>>
>>> class OtherMeta(TraceMeta):
>>> pass
>>>
>>> class SimpleDict(dict, metaclass=OtherMeta):
>>> pass
>>>
>>> class TraceDict(SimpleDict, metaclass=TraceMeta):
>>> pass
>>>
>>> trace_dict = TraceDict([('hi', 1)])
>>> trace_dict['there'] = 2
>>> trace_dict['hi']
>>> try:
>>> trace_dict['does not exist']
>>> except KeyError:
>>> pass # Expected
But this won’t work if the metaclass is from a library that I can’t modify, or if I want to use multiple utility metaclasses like TraceMeta at the same time. The metaclass approach puts too many constraints on the class that’s being modified.
To solve this problem, Python supports class decorators. Class decorators work just like function decorators: They’re applied with the @ symbol prefixing a function before the class declaration. The function is expected to modify or re-create the class accordingly and then return it:
>>> def my_class_decorator(klass):
>>> klass.extra_param = 'hello'
>>> return klass
>>>
>>> @my_class_decorator
>>> class MyClass:
>>> pass
>>>
>>> print(MyClass)
>>> print(MyClass.extra_param)
<class '__main__.MyClass'>
hello
I can implement a class decorator to apply trace_func to all methods and functions of a class by moving the core of the TraceMeta.__new__ method above into a stand-alone function. This implementation is much shorter than the metaclass version:
>>> def trace(klass):
>>> for key in dir(klass):
>>> value = getattr(klass, key)
>>> if isinstance(value, trace_types):
>>> wrapped = trace_func(value)
>>> setattr(klass, key, wrapped)
>>> return klass
I can apply this decorator to my dict subclass to get the same behavior as I get by using the metaclass approach above:
>>> @trace
>>> class TraceDict(dict):
>>> pass
>>>
>>> trace_dict = TraceDict([('hi', 1)])
>>> trace_dict['there'] = 2
>>> trace_dict['hi']
>>> try:
>>> trace_dict['does not exist']
>>> except KeyError:
>>> pass # Expected
__new__((<class '__main__.TraceDict'>, [('hi', 1)]), {}) -> {}
__getitem__(({'hi': 1, 'there': 2}, 'hi'), {}) -> 1
__getitem__(({'hi': 1, 'there': 2}, 'does not exist'), {}) -> KeyError('does not exist')
Class decorators also work when the class being decorated already has a metaclass:
>>> class OtherMeta(type):
>>> pass
>>>
>>> @trace
>>> class TraceDict(dict, metaclass=OtherMeta):
>>> pass
>>>
>>> trace_dict = TraceDict([('hi', 1)])
>>> trace_dict['there'] = 2
>>> trace_dict['hi']
>>> try:
>>> trace_dict['does not exist']
>>> except KeyError:
>>> pass # Expected
__new__((<class '__main__.TraceDict'>, [('hi', 1)]), {}) -> {}
__getitem__(({'hi': 1, 'there': 2}, 'hi'), {}) -> 1
__getitem__(({'hi': 1, 'there': 2}, 'does not exist'), {}) -> KeyError('does not exist')
When you’re looking for composable ways to extend classes, class decorators are the best tool for the job. (see Item 73: “Know How to Use heapq for Priority Queues” for a useful class decorator called functools.total_ordering.)
6.8.1. Things to Remember¶
✦ A class decorator is a simple function that receives a class instance as a parameter and returns either a new class or a modified version of the original class.
✦ Class decorators are useful when you want to modify every method or attribute of a class with minimal boilerplate.
✦ Metaclasses can’t be composed together easily, while many class decorators can be used to extend the same class without conflicts.