In this post I share some useful things I learned deploying an open source package to PyPI.
The app
I built a small PyBitesTips
class to consume our Python tips from the command line. The code (project) is here
Speaking of tips, here are some cool things I learned / re-used:
– Make a class callable using the __call__
dunder (magic) method.
– Use namedtuple
s and instantiate them with ** keyword args: [Tip(**tip) for tip in resp.json()]
– Use paging of results with pydoc.pager
.
– Break down output creation and printing in different methods (and helpers) which made testing the code easier.
Testing
I put the tests in a separate tests/
subdirectory. This way it’s easier to omit them from the package build (see further down).
I also mocked out requests.get
, providing a static tips_payload
list, and builtins.input
to simulate a user searching for various tips.
As mentioned before the abstraction of an individual tip output using _generate_tip_output
made it easy to write test_tip_output(pb_tips)
.
Another thing worth mentioning is the conftest.py
I added to the main package folder pytip
which has pytest
added to sys.path
. With that change I could just run pytest
in the top project folder (more info).
License
This is as simple as copying an existing one and updating the Copyright banner (MIT example). For more info, check out Choosing a License.
setup.py
This file is your key to making your project pip installable. As per the official documentation, a basic setuptools.setup
will do the trick. A few things to highlight as well as extra features I used for my setup.py:
- In
classifiers
you set the Python versions you support, here I use Python 3.x - In
packages
you specify which directories to bundle up, here it’s justpytip
. Note that I called itpytip
instead ofsrc
. I discovered that is how it ends up in your virtual environment’ssite-packages
. - In
install_requires
you specify any 3rd party dependencies, in this caserequests
. Note that this makesrequirements.txt
redundant, becausepython setup.py install
will now pull it in automatically. - And one of the coolest things I learned: main.py, it allows you to run your package as a console script. So here
entry_points
>console_scripts
makes an aliaspytip
that points topytip
directory (package) >__main__
module >main
function which has someargparse
logic to make this a CLI app (read much more about this in Erik’s Exploring the Modern Python Command-Line Interface). So when you pip install this package, you can just runpytip
, how cool is that, no?!
Wheels and PyPI
This is 80% of the battle. Uploading it to PyPI is actually very easy.
- Make 2 accounts: PyPI and Test PyPI.
- Get API tokens for both. Note them down because they only show them once.
-
I highly recommend making this file so you can authenticate to both servers without entering a password ever again:
$ cat ~/.pypirc [distutils] index-servers = pypi testpypi [pypi] username: __token__ password: ... [testpypi] repository: https://test.pypi.org/legacy/ username: __token__ password: ...
-
pip install
setuptools
,wheel
andtwine
. -
Create your distribution:
python setup.py sdist bdist_wheel
. This will put atar.gz
and a.whl
(wheel) in adist/
folder (which you should add to.gitignore
). -
Always first upload your package to the Test PyPI to make sure it all works:
twine upload --repository testpypi dist/*
. This is important because version numbers can only be uploaded once, so it better work before uploading it to the real PyPI. -
pip install
the package from the test index to see if you are happy with the results (here I found out about the “pytip vs src directory name” thing by looking at what got installed insite-packages
. And here I tested out if mypytip
alias, as defined inconsole_scripts
, actually worked. -
If all good, upload it to the real PyPI server:
twine upload dist/*
(no need to specifyrepository
as PyPI is the default).
And that’s it, a new open source project on PyPI.
Resources
These resources really helped me going through this process end-to-end:
- Packaging Python Projects
- How to Publish an Open-Source Python Package to PyPI – specially the console entry point / alias trick I picked up from here.
- Configuring a .pypirc File for Easier Python Packaging
“Modern” Python
Setup.py is still the way to go in many cases, but Poetry and the standardized toml
file is a strong contender.
Check it out for yourself:
Keep Calm and Code in Python!
— Bob