Click here to code!

Learning Python Decorators by Example

Posted by Bob on Fri 20 October 2017 in Concepts • 4 min read

Decorators are a sometimes overlooked feature and they might be hard to grasp for beginning Pythonistas. I agree with Aaron Maxwell that mastering them "can massively magnify the positive impact of the code you write", so make sure you add them to your toolkit if not done so already. In this article I explain what they do, why you want to use them and give some practical examples.

decorators are a bit like Russian dolls

Definition

A decorator is any callable Python object that is used to modify a function, method or class definition. A decorator is passed the original object being defined and returns a modified object, which is then bound to the name in the definition. - PythonDecorators wiki

GoF's Design Patterns defines a decorator's intent as:

Attach additional responsibilities to an object dynamically. Decorators provide a flexible alternative to subclassing for extending functionality.

Two common use cases are caching and access checks in web frameworks which I will cover later.

When to use?

If you want to add common behavior to multiple objects think about abstracting it away using decorators. It will make your code more DRY and encapsulated. It is a nice way to abstract away functionality not directly related to the function's main goal. Your team will thank you for having more reusable code.

Aaron Maxwell wrote a nice article in this context: 5 reasons you need to learn to write Python decorators.

Syntax

Python lets you decorate a function (or class) by the @ symbol followed by the decorator.

For example:

@mydecorator
def my_function(args):
    ...

Note that this is the same as:

def my_function(args):
    ...
my_function = mydecorator(my_function)

The '@' syntactic sugar is more concise and easier to read though.

Decorators can be stacked and will be run inside out:

@second_decorator
@first_decorator
def my_function(args):
    ...

This can be quite confusing so I found a good example on SO:

def makebold(fn):
    def wrapped():
        return "<b>" + fn() + "</b>"
    return wrapped

def makeitalic(fn):
    def wrapped():
        return "<i>" + fn() + "</i>"
    return wrapped

@makebold
@makeitalic
def hello():
    return "hello world"

print hello()  ## returns "<b><i>hello world</i></b>"

(now you know why I put Russian dolls in the banner)

What about passing arguments?

Expert Python provides a nice commented snippet of the complete pattern:

def mydecorator(function):
    def wrapped(*args, **kwargs):     
        # do some stuff before the original
        # function gets called
        result = function(*args, **kwargs)
        # do some stuff after function call and
        # return the result
        return result
    # return wrapper as a decorated function
    return wrapped

Make sure to add functools.wraps decorator so the original name and docstring (metadata) are not lost, specially important when debugging:

def mydecorator(function):
    @wraps(function)
    def wrapped(*args, **kwargs):
    ...

Some practical examples

I went back to our code base and found two examples where we used decorators:

Caching

For our 100 Days of Code I wrote a class to cache The Movie Database (TMDb) API responses (source):

@store_results
def get_items(self, obj_method):
    ...

decorators.py

def store_results(f):
    @wraps(f)
    def wrapped(*args, **kwargs):
        func_name = str(args[1]).lower()
        kind = re.sub(r'.*bound.*?\.(\S+) of.*', r'\1', func_name)
        print(kind)
        resp = f(*args, **kwargs)
        _store(kind, resp)
        print(len(resp))
        return resp
    return wrapped

Another caching example can be found here.

For caching / memoization you also might want to learn about @functools.lru_cache.

In computing, memoization or memoisation is an optimization technique used primarily to speed up computer programs by storing the results of expensive function calls and returning the cached result when the same inputs occur again. - Wikipedia

For Django checkout cached_property demo'd here. What's cool about it is that you can dramatically reduce making database calls improving your site's performance.

Access checking

In Never Forget A Friend’s Birthday with Python, Flask and Twilio I used a decorator to check login (source):

def login_required(test):
    '''From RealPython Flask course'''
    @wraps(test)
    def wrap(*args, **kwargs):
        if 'logged_in' in session:
            return test(*args, **kwargs)
        else:
            flash('You need to log in first')
            return redirect(url_for('login'))
    return wrap

A similar example can be found here.

Distinguishing between public and private endpoints just takes one line of extra code. It's a nice way of abstracting away the access implementation so it does not clutter and distract from writing the main Flask code:

@app.route('/login', methods=['GET', 'POST'])
def login():
    ...

@app.route('/')
@login_required
def index():
    ...

Decorators in the wild

  • See Python's Built-in Functions for some decorators that come with the Python language:

  • In the previous section, right above login_required was the all too common @app.route Flask decorator. This article explains how Flask makes it possible to write "@app.route()" at the top of the function. Another interesting discussion about this decorator and Flask's source in general can be found in The Hitchhiker's Guide to Python.

  • The Click package (Flask author) shows another elegant use of decorators.

  • Lastly take notice of mock.patch which I used here. It wraps each test method faking (mocking) the get_status Tweepy API to not hit the API while testing.

Advanced concepts

One less obvious aspect of decorators for me was the passing of optional arguments, so I wrote an article about it.

See this article for more examples of decorators that take arguments and how to decorate classes.

Practice!

The best way to learn decorators is to roll your own!

Join our decorator challenge #14 and PR your result (instructions in the challenge). You can peak at some solutions here.

Further reading

There are many good resources on decorators:


Keep Calm and Code in Python!

-- Bob


See an error in this post? Please submit a pull request on Github.


Join our community and grab our Become a Better Python Developer cheat sheet. Learn Python. Receive bonus material. Challenge yourself! (Privacy Policy)