Abstract

This PEP relaxes the constraint on dynamic metadata listed in the [project] section in pyproject.toml to allow the static portion of mixed metadata to be defined in the normal location if the field is a table or array by having the dynamic fields extend the static ones.

This allows users to opt into allowing a backend to extend metadata while still keeping the static portions of the metadata defined in the standard location in pyproject.toml, and allows inspection tools to still be able to process the static portions of the metadata.

Motivation

In the core metadata specification originally set out in PEP 621, metadata can be specified in three ways. First, it can be listed in the [project] table. This makes it statically inferable, meaning any tool (not just the build backend) can reliably compute the value. Second, a field can be listed in the project.dynamic list, which allows the build backend to compute the value. Finally, a value could be missing from both the project table and the project.dynamic list, in which case the matching metadata is guaranteed to be empty.

This system provided two important benefits to Python packaging. A standard specification that all major backends have now adopted makes teaching much easier; a single tutorial is now sufficient to cover the metadata portion of configuring any backend. Users can now switch from a general purpose backend to a specialized backend without changing their static metadata. Tooling like schema validation tools can verify and catch configuration mistakes.

The second benefit is improved support for static tools that read the source files looking for metadata. This is useful for dependency chain analysis, such as creating “used by” and “uses” graphs. It is used for code quality tooling to detect the minimum supported version of Python. It is used by cibuildwheel to automatically avoid building wheels that are not supported. It is not used, however, to avoid wheel builds when the SDist is available; that was addressed by METADATA 2.2, which a Dynamic field in the SDist metadata that lets a tool know if the metadata can change when making a wheel - this is an easy mistake to make due to the similarity of the names.

Due to the rapidly increasing popularity of the project table, support from all major backends, and a rise of backends supporting complex compiled extensions, an issue with the restrictions applied in PEP 621 is becoming more apparent. In PEP 621, the metadata choice is all-or-nothing; metadata must be completely static, or listed in the dynamic field and completely absent from the static definition. For the most common use cases, this is fine; there is little benefit to set the version statically if you are going to override it dynamically. If you are using a custom README processor to filter or modify the README for proper display, it’s not a big deal to have to specify the configuration in a custom tool.* section. But there is a specific class of cases where the all-or-nothing approach is problematic: lists of items where the backend needs to add items are currently forced to be fully dynamically specified (that is, in a backend-specific configuration location). This causes both of the original benefits (standard location and static tooling support) to be lost.

Rationale

PEP 621 includes the following statement:

In an earlier version of this PEP, tools were allowed to extend data for fields. For instance, build back-ends could take the version number and add a local version for when they built the wheel. Tools could also add more trove classifiers for things like the license or supported Python versions.

In the end, though, it was thought better to start out stricter and contemplate loosening how static the data could be considered based on real-world usage.

In this PEP, we are proposing a limited and explicit loosening of the restrictions on the [project] table and project.dynamic list.

Every list and every table with arbitrary keys will now be allowed to be specified both statically, in the [project] table, and in the project.dynamic list. If it is present in both places, the build backend can extend list items and add new keys, but not modify existing list items or strings.

[project]
dynamic = ["dependencies"]

[tool.my-build-backend]
original-dependencies = ["torch", "packaging"]
pin-to-build-versions = ["torch=={exact}"]

Dynamic: Requires-Dist
Requires-Dist: packaging
Requires-Dist: torch

Requires-Dist: packaging
Requires-Dist: torch
Requires-Dist: torch==2.8.0

[project]
dependencies = ["torch", "packaging"]
dynamic = ["dependencies"]

[tool.my-build-backend]
pin-to-build-versions = ["torch=={exact}"]

Terminology

The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

Specification

Any field that is comprised of a list or a table with arbitrary entries will now be allowed to be present in both the [project] table and the project.dynamic list. If a field is present in both places, then the build backend is allowed to insert entries into the list or table, but not remove entries, reorder entries, or modify the entries. Tables of arrays allow adding a new table entry or extending an existing array according to the rules above.

The fields that are arrays or tables with arbitrary entries are:

• authors, maintainers: New author tables can be added to the list. Existing authors cannot be modified (list of tables with pre-defined keys).
• classifiers: Classifiers can be added to the list.
• dependencies: New dependencies can be added, including more tightly constrained existing dependencies.
• entry-points: Entry points can be added, to either new or existing groups. Existing entry points cannot be changed or removed.
• scripts, gui-scripts: New scripts can be added. Existing ones cannot be changed or removed.
• keywords: Keywords can be added to the list.
• license-files: Files can be added to the list.
• optional-dependencies: A new extra or new items can be added to a existing extra.
• urls: New urls can be added. Existing ones cannot be changed or removed.
• import-names, import-namespaces: New import names or namespaces can be added. Existing ones cannot be modified or removed.

To add items, users must opt-in by listing the field in dynamic; without that, the metadata continues to be entirely static.

A backend SHOULD error if a field is specified and it does not support extending that field, to protect against possible user error. We recommend being as strict as possible to avoid unnecessary entries in the dynamic list.

Static analysis tools, when detecting a field is both specified and in the project.dynamic array, SHOULD assume the field is incomplete, allowing for new entries to be present when the package is built.

The Dynamic field, as specified in PEP 643, is unaffected by this PEP, and backends can continue to fill it as they chose. However, a backend MUST ensure that both the SDist and the wheel metadata include the static metadata portion of the project table.

Reference Implementation

The choice to support dynamic metadata for each field is already left up to backends, and this PEP simply relaxes restrictions on what a backend is allowed to do with dynamic metadata.

The pyproject-metadata project, which is used by several build backends, will need to modify the correctness check to account for the possible extensions; this is in a draft PR.

The dynamic-metadata project, which provides a plugin system that backends can use to share dynamic metadata plugins, was designed to allow this possibility, and a similar PR to the one above will allow additive metadata.

Backwards Compatibility

This does not affect any existing pyproject.toml’s, since this was strictly not allowed before this PEP.

When users adopt this in a pyproject.toml, the backend must support it; an error will be correctly generated if it doesn’t following the previous standard. Frontends were never required to throw an error, though some frontends may need to be updated to benefit from the partially static metadata. Some frontends and other tooling may need updating, such as schema validation, just like other pyproject.toml PEPs.

Using metadata from SDists or wheels is unaffected. The METADATA version does not need to be incremented.

Security Implications

There are no security concerns that are not already present, as this just adds a static component to existing dynamic metadata support.

How to Teach This

The current guides that state metadata must not be listed in both [project] and project.dynamic can be updated to say that some fields can be extended by project.dynamic. Since dynamic metadata is already an advanced concept, this will likely not affect most existing tutorial material aimed at introductory packaging.

The pyproject.toml specification will be updated to include the behavior of fields when specified and also listed in the dynamic field.

It should also be noted that specifying something in dynamic will require any tool that requires the full metadata to invoke the backend even if it is partially statically specified, so it should not be used unless necessary.

Rejected Ideas

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.