Ctrl+K

4.2 Python Project Management with Poetry

4.2 Python Project Management with Poetry#

Poetry is a tool designed to simplify the management of Python projects by handling dependency management and packaging in a unified way. Unlike traditional tools that require separate configurations for these tasks, Poetry uses a single pyproject.toml file for project specifications, making it easier to manage dependencies, package projects, and publish them to PyPI.

Benefits of Using Poetry Over Traditional Tools#

  • Simplified Dependency Management: Poetry resolves dependencies more efficiently and accurately, avoiding the common issues of dependency hell.

  • Unified Tool: Combines features of dependency management and packaging tools into one, eliminating the need for multiple configuration files.

  • Easy Version Management: Allows specifying version constraints for dependencies in a straightforward manner and handles updates smoothly.

  • Reproducible Builds: The poetry.lock file ensures that your project can be reproduced exactly the same way on any machine.

  • Intuitive CLI: Offers an easy-to-use command-line interface for managing projects, dependencies, and virtual environments.

Installing Poetry and Basic Setup#

To install Poetry, run the following command in your terminal:

curl -sSL https://install.python-poetry.org | python3 -

Initialising a New Project with Poetry#

Creating a new project is as simple as running:

poetry new my-package

Created package my_package in my-package

This command creates a new directory called my-package with a basic project structure and a pyproject.toml file pre-configured.

Project Structure#

The structure of your Python project plays a crucial role in its maintainability, scalability, and ease of testing. Two common layouts are used in the Python ecosystem: the src layout and the flat layout.

Understanding flat layout vs the src layout#

  • Flat Layout: This layout places the code that is intended to be importable (the import package) directly in the root directory of the project. While this is straightforward and commonly used in simple projects, it can lead to issues when the package tries to import itself, potentially mixing up the local code with installed versions from site-packages.

my-package-folder/
├── CHANGELOG.md             ┐
├── CODE_OF_CONDUCT.md       │
├── CONTRIBUTING.md          │
├── docs/                    │ Package documentation
│   └── ...                  │
├── LICENSE                  │
├── README.md                ┘
├── pyproject.toml           ] Package metadata and build configuration
|   my_package/              ┐
│     ├── __init__.py        │ Package source code
│     ├── moduleA.py         │
│     └── moduleB.py         ┘
└── tests/                   ┐
        └── test-file1.py    | Package tests
        └── ....             ┘

What to notice

  • my_package is in the root of the project.

  • src Layout: In the src layout, the package code lives in a subdirectory (commonly named src). This separation ensures that when you run or test the package, Python imports the installed version of the package (from site-packages) and not the local code directly. This mimics the environment that users of the package will have and can help catch potential issues related to packaging or dependencies.

my-package-folder/
├── CHANGELOG.md               ┐
├── CODE_OF_CONDUCT.md         │
├── CONTRIBUTING.md            │
├── docs                       │ Package documentation
│   └── index.md               |
│   └── ...                    │
├── LICENSE                    │
├── README.md                  ┘
├── pyproject.toml             ] Package metadata and build configuration
├── src                        ┐
│   └── my_package/            │
│       ├── __init__.py        │ Package source code
│       ├── moduleA.py         │
│       └── moduleB.py         ┘
└── tests/                     ┐
   └── ...                     ┘ Package tests

What to notice

  • my_package is not directly in the root of the project but inside the src folder.

Poetry and the project structure#

By default, when we create a new project with

poetry new my-package-src

, poetry uses flat layout. We can change to src layout with the src flag.

poetry new --src my-package-src

About cases and case conventions:#

In the course, we will use:

  • kebab case for the project name (and use it also for the remote repository name),

  • snake case for the import package name (Poetry converts the project name in kebab case to snake case).

Since valid Python identifiers cannot contain dashes (-), we should avoid using dashes in the import package names.

A valid identifier cannot start with a number, or contain any spaces and can only contain alphanumeric letters (a-z) and (0-9), or underscores (_).

Some relevant cases

  • Pascal case: e.g., NumberOfPlayers

  • Camel case: e.g., numberOfPlayers

  • Snake case: e.g., number_of_players

  • Kebab case: e.g., number-of-players

  • lowercase: e.g., numberofplayers

Converting a Pre-existing Project to Use Poetry#

Sometimes we don’t want to start from scratch but rather convert an existing project to use Poetry. To convert an existing project to Poetry, navigate to the project’s root directory and run:

poetry init

This interactive command will guide you through creating a pyproject.toml file for your project. You can specify dependencies and other project settings during this process.

References#

Python Packaging User Guide Python Packaging Authority PyOpenSci

Contents