How to handle environment variables in Python

In this article I will share 3 libraries I often use to isolate my environment variables from production code.

Why is this important?

Separate config from code

As we can read in The Twelve-Factor App / III. Config:

Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code.

https://12factor.net/config

Basically you want to be able to make config changes independently from code changes.

We also want to hide secret keys and API credentials! Notice that git is very persistent (PyCon talk: Oops, I committed my password to GitHub) so it’s important to get this right from the start.

First package: python-dotenv

These days I mostly use python-dotenv which makes this straightforward.

First install the library and add it to your requirements (or if you use Poetry it will automatically update your .toml file):

pip install python-dotenv

Secondly make an .env file with your environment variables in it.

It’s important that you ignore this file with git, otherwise you will end up committing sensitive data to your repo / project.

What I usually do is commit an empty .env-example (or .env-template) file so other developers know what they should set (see examples here and here).

So a new developer (or me checking out the repo on another machine) can do a cp .env-template .env and populate the variables. As the (checked out) .gitignore file contains .env, git won’t show it as a file to be staged for commit.

Then, to load in the variables from this file we use two lines of code:

from dotenv import load_dotenv

load_dotenv()

You can now access the environment variables using os.environ, for example:

BACKGROUND_IMG = os.environ["THUMB_BACKGROUND_IMAGE"]
FONT_FILE = os.environ["THUMB_FONT_TTF_FILE"]

To load the config without touching the environment, you can use dotenv_values(".env") which works the same as load_dotenv, except it doesn’t touch the environment, it just returns a dict with the values parsed from the .env file.

Check out the README for additional options.

Second package: python-decouple

Another library I have been using a lot with Django is python-decouple.

The process is pretty similar:

pip install python-decouple

Create an .env file with your config variables and “gitignore” it.

Then in your code you can use the config object. As per the example in the docs:

from decouple import config

SECRET_KEY = config('SECRET_KEY')
DEBUG = config('DEBUG', default=False, cast=bool)
EMAIL_HOST = config('EMAIL_HOST', default='localhost')
EMAIL_PORT = config('EMAIL_PORT', default=25, cast=int)

The casting and the ability to specify defaults are really convenient and something I miss when using python-dotenv!

Another useful option is the Csv helper. For example having this in our .env file for our platform (a Django app):

ALLOWED_HOSTS=.localhost, .herokuapp.com

We can retrieve this variable in settings.py like this:

ALLOWED_HOSTS = config('ALLOWED_HOSTS', cast=Csv())

Third package: dj-database-url

And while we are here, there is one more package I want to show you: dj-database-url, which makes it easier to load in your database URL.

As per the docs:

The dj_database_url.config method returns a Django database connection dictionary, populated with all the data specified in your URL. There is also a conn_max_age argument to easily enable Django’s connection pool.

https://pypi.org/project/dj-database-url/

And here is how to use it:

import dj_database_url

DATABASES = {
    'default': dj_database_url.config(
        default=config('DATABASE_URL')
    )
}

Nice and clean!

This is what I mostly use, for more options, check out python-decouple‘s README here.

Python Tips

As a recap, here is the python-decouple code in a concise tip you can easily paste into your project:

# pip install python-decouple dj-database-url

from decouple import config, Csv
import dj_database_url

SECRET_KEY = config('SECRET_KEY')
DEBUG = config('DEBUG', default=False, cast=bool)
ALLOWED_HOSTS = config('ALLOWED_HOSTS', cast=Csv())

DATABASES = {
    'default': dj_database_url.config(
        default=config('DATABASE_URL')
    )
}

We also did a video on this here:

We love practical tips like these, to get our growing collection check out our book: PyBites Python Tips – 250 Bulletproof Python Tips That Will Instantly Make You A Better Developer

And with that we got a wrap. I hope this has been useful and will make it easier for you to separate config from code, which I wholeheartedly agree with The Twelve-Factor App, is important.

— Bob