Submitting a Python package with GitHub and PyPI
The submission process has changed slightly since I wrote this post.
PyPI is migrating from https://pypi.python.org/pypi/ to http://pypi.org. The old site is still up, but I don’t think it’s accepting submissions anymore - something I discovered while trying to submit a new version of simplestatistics.
The two big differences are:
- The submission url has changed. If you have a
~/pypircfile, you can probably fix the submission issue by removing a line that begins with
respository=, allowing the default url to be used. See this post for more information.
- You used to be able to specify a different url in
testpypi, the test server you could use to test the submission. I think this no longer works, and you might need to use twine to be able to make use of that again. I think PyPI is generally encouraging everyone to use twine for all their submission needs.
Note: The order of steps matters.
Note 2: Some of these steps are specific to hosting your package on GitHub.
- Do you have a
- Update version number in documentation
- Update version number in
- Add tarball download url to
- If there are new files that should be included, edit
- Commit all those changes. Have a clean repo.
- Add/create a git tag
- Push git tag to remote
- Confirm that GitHub has generated the release file
- Release testing
- Add changelog notes to GitHub release page/tag
- See also
Do you have a
You need to create a setup script to publish your package using
distutils. This is the one for
from distutils.core import setup setup( name = 'simplestatistics', packages = ['simplestatistics', 'simplestatistics.statistics'], version = '0.2.5', description = 'Simple statistical functions implemented in readable Python.', author = 'Sherif Soliman', author_email = 'email@example.com', copyright = 'Copyright (c) 2016 Sherif Soliman', url = 'https://github.com/sheriferson/simplestatistics', download_url = 'https://github.com/sheriferson/simplestatistics/tarball/0.2.5', keywords = ['statistics', 'math'], classifiers = [ 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 3', 'Topic :: Scientific/Engineering :: Mathematics', 'Intended Audience :: Developers', 'Intended Audience :: Education', 'Intended Audience :: End Users/Desktop', 'Intended Audience :: Science/Research', 'Operating System :: MacOS', 'Operating System :: Unix', 'Topic :: Education', 'Topic :: Utilities' ] )
If you maintain a changelog file, make sure you update it with the release version and date.
If you don’t maintain a changelog file, you should.
I maintain a separate
HISTORY.rst file, so I make sure I update that too.
Update version number in documentation
Good documentation is important to me. It helps you understand your code and project better, and definitely helps anyone else trying to use it. simplestatistics documentation is hosted and generated automatically by Read the Docs using the very useful Sphinx package.2
When I’m publishing a new release, I make sure I update the version number in Sphinx’s
Update version number in
setup.py in the first step. Make sure to update the version number.
from distutils.core import setup setup( ... version = '0.2.5', ...
PyPI doesn’t like Markdown. Actually, it’s not that it doesn’t like it, it just doesn’t care about it one way or another. PyPI likes reStructuredText (RST from this onwards). If you want PyPI to render the README on the package homepage like you can see on the simplestatistics PyPI page, it has to be in RST.
I’ve had a lot of trouble with RST. In my experience, it’s very fragile. It takes one extra space in a table to break rendering for the whole file.
pandoc is a great tool you could use to convert your
README.rst, but PyPI may not like the default output of pandoc’s conversion. In my use case, the conversion of the Markdown tables to RST tables was the part that often angered PyPI rendering.
After some troubleshooting, I found that this command prevents the tables from wrapping around to new lines and causing README rendering on PyPI to fail.
pandoc --columns=100 --output=README.rst --to rst README.md
Here’s a bonus tip: this online reStructuredText editor, made available by Andrey Rublev, has been a huge help in debugging and fixing RST errors.
Add tarball download url to
In a later step, we will add a git tag and push it to GitHub. This will create a new release on the GitHub page, and this release will include
.tar.gz files of the release (see an example here).
These compressed files are the ones that PyPI will pull from when you push your release or update. PyPI gets that download url from
The result of this circle is that you need to anticipate the url for the release on GitHub before you push the release commit to GitHub.
You can set the new download url in
setup.py based on your new version number:
setup( ... url = 'https://github.com/sheriferson/simplestatistics', download_url = 'https://github.com/sheriferson/simplestatistics/tarball/0.2.5', ...
Yes, you do set this url and commit it before it actually exists.
If there are new files that should be included, edit
MANIFEST.in file is how you tell
disutils to include files in the release file that it wouldn’t include otherwise. This is my
include LICENSE.txt include README.rst HISTORY.rst
Commit all those changes. Have a clean repo.
Commit everything we’ve done so far. Have a consistent commit comment for those changes. I usually add the message: “Prep for 0.2.5 release.”
Add/create a git tag
git tag 1.2.3 -m "Adds 1.2.3 tag for PyPI
Once you have the project in the state you want for creating the release, you add a git tag with the version number of the release. This will be reflected in the “releases” page of your GitHub repository.
Push git tag to remote
git push --tags origin master
Push those tags to GitHub.
Confirm that GitHub has generated the release file
Browse to your releases page (example) and make sure the new version has a release entry with its corresponding files.
See update at the top for notes about testpypi. The instructions below are here for posterity only. You probably want to use twine.
python setup.py register -r pypitest
You are, or aspire to be a good programmer who wants to be as cautious as possible, and so you’d like to test releasing the update on PyPI before actually doing it.
PyPI provides a test system that you can use to test the registration, upload, and installation of your package. Let’s make use of that gift.
python setup.py register -r pypitest will register the package on the
If you get a message indicating the all-clear, continue.
python setup.py sdist upload -r pypitest
… will upload the distribution of your package to
If something in the package is broken, you cannot make changes and reupload the package with the same version number. This means that if you want to make a change or fix something, you will have to change the version number in
setup.py (and accordingly everywhere else) and start all over again.3
pip install -i https://testpypi.python.org/pypi simplestatistics
… is the final step in the process of testing the release. This will try to install the new version of the package from
pypi. If all goes well, you should be able to import the package normally in the REPL or in a Python script.
Once you’ve tested importing the package, it’s a good idea to
pip uninstall your package so you can test the installation from the live PyPI servers.
python setup.py register -r pypi
You are finally at the actual release stage. This will register the new version with PyPI.
python setup.py sdist upload -r pypi
… uploads the distribution to PyPI.
pip install simplestatistics
… tests the installation from PyPI. This is why we
pip uninstalled the version from
Add changelog notes to GitHub release page/tag
This is not necessary, but it’s good practice and shows care for maintaining documentation of your open source project. Edit the new GitHub release and add notes about what changed in nicely formatted Markdown.
You made it. It feels good to have a package on PyPI. It helps you use your own package in the future, and it’s a good contribution to make it available to everyone else. Go celebrate.
- How to submit a package to PyPI - I learned a large part of what I described here from this helpful article by Peter Downs. It’s a good resource.
Which it is!
pip install simplestatistics↩︎
The thing I like most about Sphinx is that you write the code, and in the case of
simplestatisticsthe tests, in the docstrings of each
.pyfile. The thing I like the least is that you have to use reStructured text, which is a bouquet of sadness. ↩︎
At the time of writing. Things might change and become more flexible in the future. I hope so. ↩︎