Avoiding Silent Failures in Python: Best Practices for Error Handling

In the world of programming, errors are inevitable. But how we choose to handle these errors can make the difference between a system that is robust and user-friendly and one that is fraught with ambiguous issues 😱

The Zen of Python famously states, “Errors should never pass silently.” This principle emphasizes the importance of addressing issues head-on rather than ignoring them 😅

In this article, we’ll delve into the consequences of silent failures, the value of transparent error handling, and practical ways to ensure that errors in our Python code are always brought to light.

Requests

Let’s look at a practical example:

# 1. silencing an error
import requests

def get_data(url):
    try:
        response = requests.get(url)
        data = response.json()
    except:  # should always name exceptions. 
        data = {}
    return data

# 2. be explicit and raise the issue
def get_data(url):
    response = requests.get(url)
    # raise an HTTPError if the HTTP request returned an unsuccessful status code  
    response.raise_for_status() 
    data = response.json()
    return data

When working with APIs, getting a non-200 (OK) response usually means something went wrong.

Silently handling these failures might lead to ambiguous situations.

In the code above, the function attempts to retrieve data from a given URL. If anything goes wrong it silently returns an empty dictionary 🤔

This could mask potential problems and make the root cause harder to find.

In the refactored version, we explicitly use response.raise_for_status(), which will raise an HTTPError if the HTTP request returned an unsuccessful status code.

This means the failure is made explicit and can be handled appropriately by the caller, preventing bugs that might otherwise be difficult to trace 💡

Silencing errors

Another thing you might see in Python code is silencing of errors, for example:

try:
    # some code block
except SomeException:
    pass

Or even worse:

try:
    # some code block
except:  # exception not explicitly named :(
    pass

Here, if an exception is raised inside the try block, the except block will execute. However, because it only contains the pass statement, nothing will actually happen. The error will be silently ignored 😱, and the program will continue executing subsequent lines of code as if nothing went wrong 😅

It’s advisable to handle the error, any error. At the very least, log it.

Overall, in software it’s usually better to fail fast than to hide an error which causes a bug down the line that now is further away from the original source / cause.

Note that if you really want to mute an error (after all, there is an “Unless explicitly silenced” part in the Zen of Python), you can do that explicitly with the suppress context manager from the contextlib module:

import os
from contextlib import suppress

with suppress(FileNotFoundError):
    os.remove('somefile.tmp')

Zip truncating

Another example is the zip() built-in’s recent addition of “strict” (Python 3.10).

Zip()’s default behavior is still to truncate the longer sequence if the shorter sequence ends, for example:

>>> names = "ana sara bob julian".split()
ages = (11, 22, 33)
>>> list(zip(names, ages))
[('ana', 11), ('sara', 22), ('bob', 33)]

Poor Julian’s entry got truncated. No error was raised.

But you can now:

>>> list(zip(names, ages, strict=True))
...
ValueError: zip() argument 2 is shorter than argument 1

This might blow up your program but at least we give an early indication that something is off, in this case the second iterable being shorter.

Note that you can gracefully handle this by using itertools’ zip_longest() that lets you give a default value for items of the longer sequence:

>>> from itertools import zip_longest
>>> list(zip_longest(names, ages))
[('ana', 11), ('sara', 22), ('bob', 33), ('julian', None)]
>>> list(zip_longest(names, ages, fillvalue=10))
[('ana', 11), ('sara', 22), ('bob', 33), ('julian', 10)]

This is Pybites Tip 137 actually – for 249 more real world Python tips, check out our book 😎

Conclusion

Throughout this article, the consistent theme is that errors should be exposed and addressed, not hidden.

A fail fast scenario is often better than letting an error propagate through the system.

So:

1. Check the code you’re using if you can have it fail early and loudly and handle the exception.

2. Think about the code you write, consider potential silent errors, and handle those edge cases as soon as possible so they don’t cause a mess further down the line. 

Check error conditions early

I recently looked at some code on GitHub and in this example the author clearly does not want both the min and max input variables to be None, and sets clear expectations high up in the function:

  if min is None and max is None:
    raise AssertionError(
      'At least one of `min` or `max` must be specified.'
    )

… not letting an error condition pass silently 👍