A debugging tale

I ran into an interesting issue debugging the other day …

I used isort with pre-commit to automatically sort imports before committing my code.

This is a huge time saver and I am very thankful for both tools, as well as black and flake8.

They save a ton of time and they instantly boost the quality of your code fixing easy-to-avoid mistakes shaving off debugging time.

I did a walk-through video the other day on how to use pre-commit here.

One thing that puzzled me though was isort’s behavior on the following test module:

# 01/test_app.py

import pytest
from fastapi.testclient import TestClient

from app import app

When running isort through pre-commit it sorted it like:

import pytest
from app import app  # wtf?
from fastapi.testclient import TestClient

Which is not correct because app.py is my own module, not a third party one!

So I did a bit of debugging and I thought it would be interesting to share my approach here.

Reproducible case

Before doing anything else I checked for the easiest “reproducible case” to get more info from the program repeatedly. This is an essential debugging technique I picked up over the years.

In this case it meant running isort by itself (outside of pre-commit) and adding the -c and -v flags to it.

The -c flag does not edit the file in place (hence easy to repeatedly reproduce the issue). The -v flag adds useful information to isort’s command line output:

So here we see the error in action: isort marked app.py unrightfully as THIRDPARTY.

Apart from that key insight I learned about isort’s messaging which brings me to the next step …

Debugging the source code

I checked out the isort repo and used ag (a.k.a. the Silver Searcher, one of my favorites) to look for from-type place_module:

(Update: the day after writing this article I learned that git grep -n <term> achieves the same thing as ag without the extra dependency – thanks Jeff!)

That led me to isort’s parse.py where I added a breakpoint() to drop into the pdb debugger:

It’s important to note that the isort I had been using so far was “globally” installed with pipx:

$ which isort
/Users/bbelderbos/.local/bin/isort

So to use the locally checked out isort version (with the breakpoint in it), I used this command (which worked because there is a __main__.py package entry point to the package):

$ python -m isort ~/code/fastapi-sqlmodel/01 -c -v
…

I soon found out that finder was finder = partial(place.module, config=config) so I went one level deeper to the place.py module and ended up inspecting this particular code:

I learned that all these conditionals were False for app.py so isort hit the last condition for my “app” module:

(config.default_section, "Default option in Config or universal default.")

That gave me a hint that this might have been a config issue all along.

This brought me to isort’s config options where I learned that default_section is THIRDPARTY by default.

At least that explained the erroneous result. Digging a bit deeper I learned that you can actually force isort to recognize a module as “first party” with the -p flag:

But I felt this was merely solving the symptom, not the cause!

Going back to the module_with_reason checks, I suspected that one of conditions these being False could hint at a setup issue on my end (I really trust isort after all!)

Looking at the _local() helper gave me the final hint:

Ah! Is this an import path issue perhaps?!

And realizing (🤦) that I was actually running isort against 01/test_app.py, the next check was to cd into the 01 directory and try it again:

As I need these subdirectories because I am working on a PyBites Learning Path, I looked at an easy config fix for this …

I now had a way more targeted search string for Google thanks to the debugging effort:

isort import one level directory down

First hit.

What stood out quickly glazing over the first comment was --known-local-folder

Second Google search:

pre-commit isort allow relative imports

This brought me back to the beforementioned config page.

It turns out you can set this in your isort config:

.isort.cfg
[settings]
known_local_folder=my_module1,my_module2

So I adapted that to my needs:

.isort.cfg
[settings]
known_local_folder=app

And then it all worked:

As most Bites will use app.py this will probably do as a fix, but if a future module will be called api.py, I can easily add it to this config file.

As this was not the only file it had failed on, as a last step I could retroactively fix it for all files using pre-commit’s --all-files option: pre-commit run --all-files > nice!

Lessons learned / takeaways from this story:

1. Find an easy-and-fast-to-reproducible use case of the issue, here: run isort outside of pre-commit + use -c to not modify files in-place.
   
2. Make your tools as verbose as possible, here this was easy: isort -v
   
3. With the messaging go to the source and see what it is doing, breakpoint() / pdb for the win!
   
4. With the things you picked up debugging, you can search Google in a way more targeted manner.
   
5. Bonus: I got to know isort a little bit better. Any time you spend on reading code usually has a great ROI!