How to Deploy Your Open Source Package to PyPI

Bob, Mon 31 August 2020, Packages

classes, opensource, packaging, pypi, setup.py, tips

In this post I share some useful things I learned deploying an open source package to PyPI.

The app

I built a small PyBitesTips class to consume our Python tips from the command line. The code (project) is here

Speaking of tips, here are some cool things I learned / re-used: - Make a class callable using the __call__ dunder (magic) method. - Use namedtuples and instantiate them with ** keyword args: [Tip(**tip) for tip in resp.json()] - Use paging of results with pydoc.pager. - Break down output creation and printing in different methods (and helpers) which made testing the code easier.

Testing

I put the tests in a separate tests/ subdirectory. This way it's easier to omit them from the package build (see further down).

I also mocked out requests.get, providing a static tips_payload list, and builtins.input to simulate a user searching for various tips.

As mentioned before the abstraction of an individual tip output using _generate_tip_output made it easy to write test_tip_output(pb_tips).

Another thing worth mentioning is the conftest.py I added to the main package folder pytip which has pytest added to sys.path. With that change I could just run pytest in the top project folder (more info).

License

This is as simple as copying an existing one and updating the Copyright banner (MIT example). For more info, check out Choosing a License.

setup.py

This file is your key to making your project pip installable. As per the official documentation, a basic setuptools.setup will do the trick. A few things to highlight as well as extra features I used for my setup.py:

Wheels and PyPI

This is 80% of the battle. Uploading it to PyPI is actually very easy.

  1. Make 2 accounts: PyPI and Test PyPI.
  2. Get API tokens for both. Note them down because they only show them once.
  3. I highly recommend making this file so you can authenticate to both servers without entering a password ever again:

    $ cat ~/.pypirc
    [distutils]
    index-servers =
        pypi
        testpypi
    
    [pypi]
    username: __token__
    password: ...
    
    [testpypi]
    repository: https://test.pypi.org/legacy/
    username: __token__
    password: ...
    
  4. pip install setuptools, wheel and twine.

  5. Create your distribution: python setup.py sdist bdist_wheel. This will put a tar.gz and a .whl (wheel) in a dist/ folder (which you should add to .gitignore).

  6. Always first upload your package to the Test PyPI to make sure it all works: twine upload --repository testpypi dist/*. This is important because version numbers can only be uploaded once, so it better work before uploading it to the real PyPI.

  7. pip install the package from the test index to see if you are happy with the results (here I found out about the "pytip vs src directory name" thing by looking at what got installed in site-packages. And here I tested out if my pytip alias, as defined in console_scripts, actually worked.

  8. If all good, upload it to the real PyPI server: twine upload dist/* (no need to specify repository as PyPI is the default).

And that's it, a new open source project on PyPI.

Resources

These resources really helped me going through this process end-to-end:

"Modern" Python

Setup.py is still the way to go in many cases, but Poetry and the standardized toml file is a strong contender.

Check it out for yourself:


Keep Calm and Code in Python!

-- Bob