How to Automate Python Package Publishing with GitHub Actions

Contents show

TL;DR:

Publishing Python packages can be daunting, but automation can simplify the process significantly. This guide walks you through setting up GitHub Actions to automate publishing your Python package to PyPI. From version tagging and workflow setup to using trusted publishers for secure deployment, we’ll cover every step. By the end, you’ll have a fully automated process, including an optional Homebrew integration for macOS users.

Introduction:

Publishing a Python package is essential for making your code accessible to the community, but the process can feel overwhelming. Between version management, creating GitHub releases, and updating PyPI, there’s a lot to handle.

In this guide, we’ll show you how to automate the entire workflow using GitHub Actions. You’ll learn how to manage version tags, use trusted publishers for secure authentication, and publish packages seamlessly. By the end, you’ll have a robust pipeline to streamline your package releases.

Setting Up Python Package Publishing

1. Register on PyPI

Before automating the process, register an account on PyPI and create a project with a unique name. For example, if your package accesses the Paypal API, you might name it “PaypalConnector.”

2. Configure Trusted Publishers

PyPI’s trusted publisher mechanism allows secure integration with third-party services like GitHub.

Go to the “Publishing” section in your PyPI project settings.
Select GitHub as the trusted publisher.
Add your repository and GitHub workflow details (e.g., release.yml).

This approach is more secure than using API keys, as it relies on short-lived tokens.

3. GitHub Repository Setup

Organize Your Code:
- Place your package code in a repository.
- Include essential files like setup.py or pyproject.toml for package configuration.
Create Workflow Files:
- Navigate to .github/workflows/ in your repo.
- Create a file named release.yml.

GitHub Actions Workflow Breakdown

Triggering the Workflow

The workflow runs when you push a tag that matches the Python versioning standard (PEP 440). For example, tags like v1.0.0 or v1.2.3 will trigger the action.

Job 1: Extract Details

Extract key details from the version tag:

Version Number: Ensures compatibility with PyPI.
Outputs: Pass extracted data (e.g., version number) to subsequent jobs.

Job 2: Check PyPI for Conflicts

Fetch the latest published version from PyPI to avoid duplication.

If a version already exists, terminate the workflow.
PyPI does not allow re-publishing the same version, even if deleted.

Job 3: Build the Package

Use Poetry to build your package:

Setup Python Environment: Install dependencies and prepare the build environment.
Build Artifacts: Generate the source and wheel distributions, stored in a dist/ folder.

Job 4: Publish to PyPI

Publish the package using PyPI’s trusted publisher mechanism:

Use the ID token permissions provided by GitHub Actions.
Upload the artifacts to PyPI.

Job 5: Create a GitHub Release

Create a release on GitHub with the new version tag:

Include a changelog and artifacts.
This step makes your release more visible to users and contributors.

Releasing New Versions

Releasing a new version becomes a breeze:

Update your codebase and increment the version (e.g., 0.0.2).
Commit changes and push a new tag (git tag v0.0.2 && git push origin tags).
GitHub Actions will handle the rest.

Optional Bonus: Publish to Homebrew

For macOS users, you can extend the workflow to publish a CLI package to Homebrew:

Create a separate GitHub repository for your Homebrew tap.
Define a formula in Ruby to describe your package.
Use GitHub Actions to update the tap repository with the latest package version.

Best Practices for Automation

Follow PEP 440 Standards:
Use proper versioning to avoid issues with PyPI.
Secure Your Workflow:
- Use trusted publishers instead of API keys.
- Regularly review permissions for tokens and repositories.
Test Before Publishing:
Always test your workflows in a staging environment to catch errors early.

Conclusion:

Automating your Python package publishing workflow can save time, reduce errors, and make releases more consistent. By using GitHub Actions, trusted publishers, and proper versioning, you can streamline the process from code changes to PyPI deployment.

Ready to simplify your package releases? Start by setting up your first GitHub Actions workflow and share your experience in the comments below.