5.3 Packaging

In this chapter, we’ll explore the essentials of Python distribution packages, package registries, and how to use uv to build and publish Python packages. Additionally, we will delve into automating the build and publish process using the GitHub Actions CI/CD pipeline.

Package Registries¶

A Package Registry is a storage space for packages where they can be published, shared, and managed. It allows developers to easily distribute and install packages using package management tools. The Python Package Index (PyPI) is the standard registry for Python packages. Organizations may also use private registries for internal tools and libraries.

When you use a command like pip install package-name, pip looks for the package in the configured registry (by default, PyPI) and installs it. It is also possible to use a different registry by specifying it using the --index-url option with pip. For example, pip install --index-url https://test.pypi.org/simple/ package-name will install the package from TestPyPI instead of the main PyPI registry.

Although GitHub Packages allows you to host and manage packages within your GitHub repository, unfortunately, it does not support Python packages.

Therefore, to distribute Python packages, we will use the Python Package Index (PyPI). To avoid publishing a package to the main PyPI registry during testing, we can use TestPyPI, which is a separate instance of the Python Package Index.

Creating an account on TestPyPI¶

To create an account on TestPyPI, follow these steps:

1. Go to https://test.pypi.org/account/register/

2. Fill in the registration form with your details.

3. Verify your email address by clicking on the verification link sent to your email.

To use TestPyPI, you will also need to enable 2FA on TestPyPI. Follow these steps to enable 2FA:

1. Go to https://test.pypi.org/manage/account/

2. Enable two-factor authentication.

3. Save your recovery codes.

Building our package¶

To publish a package to TestPyPI, you need to create a package distribution file. This file contains the package’s code, resources, and metadata, and can be installed using package management tools. uv offers a command to build the package distribution files: uv build.

Publishing our package¶

To publish a package to TestPyPI, we can use an official action provided by the Python Packaging Authority (PyPA) called gh-action-pypi-publish. This action allows you to publish your package to PyPI or TestPyPI directly from your GitHub repository. You can configure the action to publish to TestPyPI by setting the repository_url input to https://test.pypi.org/legacy/.

Automating Build and Publish with GitHub Actions¶

Secure Authentication with Trusted Publishing¶

TestPyPI requires secure authentication. The easiest modern approach is Trusted Publishing with OIDC.

How it works:

1. GitHub Actions requests an identity token.

2. TestPyPI checks whether that workflow is allowed to publish your project.

3. TestPyPI creates a short-lived upload token.

4. Your artifacts are uploaded without storing a permanent secret in GitHub.

This is better than saving API tokens in repository secrets.

Register the GitHub workflow as a trusted publisher in TestPyPI¶

TestPyPI does not allow to users to publish packages with the same name. Therefore, the name of the package you want to publish must be unique.

• The name in [project].name must match the project you want to publish.

• The distribution name and the repository setup should stay consistent.

• If you rename the package later, you must also update the Trusted Publisher settings.

If the TestPyPI project does not exist yet:

1. Go to https://test.pypi.org/manage/account/publishing/

2. Add a pending publisher.

If the TestPyPI project already exists:

1. Go to https://test.pypi.org/manage/project/my-package/publishing/

2. Add a publisher there.

Use these values:

• Project name: your package name from pyproject.toml

• Owner: your GitHub username or organization

• Repository name: your GitHub repository name

• Workflow name: release.yml

• Environment: leave empty unless you intentionally use GitHub environments

Adding the build and publish steps to GitHub Actions¶

To automate the build we just need to add a build step to the workflow. We will also use the actions/upload-artifact action to upload the distribution files as artifacts of the workflow, so we can download them later if needed.

      - name: Build distributions
        run: uv build
        
      - name: Upload dist artifacts
        uses: actions/upload-artifact@main
        with:
          name: dist
          path: dist/
          if-no-files-found: error

We can now commit and push the changes to the repository. If the commit message follows the Conventional Commits format, a release will be created and the build step will be executed. We can check that the artifacts are uploaded in the “Actions” tab of the repository, under the workflow run.

To publish the package, we just need to add the following step:

      - name: Publish to TestPyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          repository-url: https://test.pypi.org/legacy/

and in the permissions section of the workflow, we need to add a permission for id-token: write to allow the workflow to request an identity token for authentication with TestPyPI:

    permissions:
      contents: write
      id-token: write

Verify the package is published¶

After a release, confirm package availability at:

https://test.pypi.org/project/my-package-with-semver/

Install test command:

pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ my-package-with-semver

The --extra-index-url option allows pip to also search the main PyPI registry for dependencies that may not be available on TestPyPI.

If other developers would want to add your package as a dependency using uv, they can add it using the command:

uv add --index testpypi=https://test.pypi.org/simple/ my-package-with-semver

Common issues¶

Issue: Invalid trusted publisher

• Cause: the owner, repository, or workflow name does not exactly match TestPyPI.

• Fix: compare the TestPyPI Trusted Publisher settings to your GitHub repository and workflow file.

Issue: No OIDC token available

• Cause: the job does not have id-token: write permission.

• Fix: add id-token: write to the publishing job.

Issue: no release happens

• Cause: commits do not follow the Conventional Commits format.

• Fix: use fix:, feat:, or a breaking-change commit message.

Issue: File already exists

• Cause: that exact version was already published.

• Fix: publish a new version instead of retrying the same file set.