Abstract

This PEP defines a specification for how licenses are documented in the core metadata, with license expression strings using SPDX identifiers in a new License-Expression field. This will make license declarations simpler and less ambiguous for package authors to create, end users to read and understand, and tools to programmatically process.

The PEP also:

• Formally specifies a new License-File field, and defines how license files should be included in distributions, as already used by the Wheel and Setuptools projects.
• Deprecates the legacy License field and license :: classifiers.
• Adds and deprecates the corresponding keys in the pyproject.toml [project] table.
• Provides clear guidance for authors and tools converting legacy license metadata, adding license files and validating license expressions.
• Describes a reference implementation and analyzes numerous potential alternatives.

The changes in this PEP will update the core metadata to version 2.4, modify the project (source) metadata specification, and make minor additions to the source distribution (sdist), built distribution (wheel) and installed project standards.

Goals

This PEP’s scope is limited to covering new mechanisms for documenting the license of a distribution package, specifically defining:

• A means of specifying a SPDX license expression.
• A method of including license texts in distributions and installed projects.

The changes to the core metadata specification that this PEP requires have been designed to minimize impact and maximize backward compatibility. This specification builds off of existing ways to document licenses that are already in use in popular tools (e.g. adding support to core metadata for the License-File field already used in the Wheel and Setuptools projects) and by some package authors (e.g. storing an SPDX license expression in the existing License field).

In addition to these proposed changes, this PEP contains guidance for tools handling and converting these metadata, a tutorial for package authors covering various common use cases, detailed examples of them in use, and a comprehensive survey of license documentation in Python and other languages.

It is the intent of the PEP authors to work closely with tool maintainers to implement the recommendations for validation and warnings specified here.

Non-Goals

This PEP is neutral regarding the choice of license by any particular package author. This PEP makes no recommendation for specific licenses, and does not require the use of a particular license documentation convention.

Rather, the SPDX license expression syntax proposed in this PEP provides a simpler and more expressive mechanism to accurately document any kind of license that applies to a Python package, whether it is open source, free/libre, proprietary, or a combination of such.

This PEP also does not impose any additional restrictions when uploading to PyPI, unless projects choose to make use of the new fields.

Instead, it is intended to document best practices already in use, extend them to use a new formally-specified and supported mechanism, and provide guidance for packaging tools on how to hand the transition and inform users accordingly.

This PEP also is not about license documentation in files inside projects, though this is a surveyed topic in an appendix, and nor does it intend to cover cases where the source and binary distribution packages don’t have the same licenses.

Motivation

Software must be licensed in order for anyone other than its creator to download, use, share and modify it, so providing accurate license information to Python package users is an important matter. Today, there are multiple fields where licenses are documented in core metadata, and there are limitations to what can be expressed in each of them. This often leads to confusion and a lack of clarity, both for package authors and end users.

Many package authors have expressed difficulty and frustrations due to the limited capabilities to express licensing in project metadata, and this creates further trouble for Linux and BSD distribution re-packagers. This has triggered a number of license-related discussions and issues, including on outdated and ambiguous PyPI classifiers, license interoperability with other ecosystems, too many confusing license metadata options, limited support for license files in the Wheel project, and the lack of clear, precise and standardized license metadata.

The current license classifiers address some common cases, and could be extended to include the full range of current SPDX identifiers while deprecating the many ambiguous classifiers (including some popular and problematic ones, such as License :: OSI Approved :: BSD License). However, this requires a substantial amount of effort to duplicate the SPDX license list and keep it in sync. Furthermore, it is effectively a hard break in backward compatibility, forcing a huge proportion of package authors to immediately update to new classifiers (in most cases, with many possible choices that require closely examining the project’s license) immediately when PyPI deprecates the old ones.

Furthermore, this only covers simple packages entirely under a single license; it doesn’t address the substantial fraction of common projects that vendor dependencies (e.g. Setuptools), offer a choice of licenses (e.g. Packaging) or were relicensed, adapt code from other projects or contain fonts, images, examples, binaries or other assets under other licenses. It also requires both authors and tools understand and implement the PyPI-specific bespoke classifier system, rather than using short, easy to add and standardized SPDX identifiers in a simple text field, as increasingly widely adopted by most other packaging systems to reduce the overall burden on the ecosystem. Finally, this does not provide as clear an indicator that a package has adopted the new system, and should be treated accordingly.

On average, Python packages tend to have more ambiguous and missing license information than other common ecosystems (such as npm, Maven or Gem). This is supported by the statistics page of the ClearlyDefined project, an Open Source Initiative incubated effort to help improve licensing clarity of other FOSS projects, covering all packages from PyPI, Maven, npm and Rubygems.

Rationale

A survey of existing license metadata definitions in use in the Python ecosystem today is provided in an appendix of this PEP, and license documentation in a variety of other packaging systems, Linux distros, languages ecosystems and applications is surveyed in another appendix.

There are a few takeaways from the survey, which have guided the design and recommendations of this PEP:

• Most package formats use a single License field.
• Many modern package systems use some form of license expression syntax to optionally combine more than one license identifier together. SPDX and SPDX-like syntaxes are the most popular in use.
• SPDX license identifiers are becoming the de facto way to reference common licenses everywhere, whether or not a full license expression syntax is used.
• Several package formats support documenting both a license expression and the paths of the corresponding files that contain the license text. Most Free and Open Source Software licenses require package authors to include their full text in a distribution.

The use of a new License-Expression field will provide an intuitive, structured and unambiguous way to express the license of a package using a well-defined syntax and well-known license identifiers. Similarly, a formally-specified License-File field offers a standardized way to ensure that the full text of the license(s) are included with the package when distributed, as legally required, and allows other tools consuming the core metadata to unambiguously locate a distribution’s license files.

While dramatically simplifying and improving the present Python license metadata story, this specification standardizes and builds upon existing practice in the Setuptools and Wheel projects. Furthermore, an up-to-date version of the current draft of this PEP is already successfully implemented in the popular PyPA Hatch packaging tool, and an earlier draft of the license files portion is implemented in Setuptools.

Over time, encouraging the use of these fields and deprecating the ambiguous, duplicative and confusing legacy alternatives will help Python software publishers improve the clarity, accuracy and portability of their licensing practices, to the benefit of package authors, consumers and redistributors alike.

Terminology

This PEP seeks to clearly define the terms it uses, given that some have multiple established meanings (e.g. import vs. distribution package, wheel format vs. Wheel project); are related and often used interchangeably, but have critical distinctions in meaning (e.g. [project] key vs. core metadata field); are existing concepts that don’t have formal terms/definitions (e.g. project/source metadata vs. distribution/built metadata, build vs. publishing tools), or are new concepts introduced here (e.g. license expression/identifier).

This PEP also uses terms defined in the PyPA PyPUG Glossary (specifically built/binary distribution, distribution package, project and source distribution), and by the SPDX Project (license identifier, license expression).

The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

Terms are listed here in their full versions; related words (Rel:) are in parenthesis, including short forms (Short:), sub-terms (Sub:) and common synonyms for the purposes of this PEP (Syn:).

Core Metadata (Syn: Package Metadata, Sub: Distribution Metadata)

The PyPA specification and the set of metadata fields it defines that describe key static attributes of distribution packages and installed projects.

The distribution metadata refers to, more specifically, the concrete form core metadata takes when included inside a distribution archive (PKG-INFO in a sdist and METADATA in a wheel) or installed project (METADATA).

Core Metadata Field (Short: Metadata Field/Field)

A single key-value pair, or sequence of such with the same key, as defined by the core metadata specification. Notably, distinct from a pyproject.toml [project] table key.

Distribution Package (Sub: Package, Distribution Archive)

(See PyPUG) In this PEP, package is used to refer to the abstract concept of a distributable form of a Python project, while distribution more specifically references the physical distribution archive.

License Classifier

A PyPI Trove classifier (as described in the core metadata specification) which begins with License ::, currently used to indicate a project’s license status by including it as a Classifier in the core metadata.

License Expression (Syn: SPDX Expression)

A string with valid SPDX license expression syntax including any SPDX license identifiers as defined here, which describes a project’s license(s) and how they relate to one another. Examples: GPL-3.0-or-later, MIT AND (Apache-2.0 OR BSD-2-clause)

License Identifier (Syn: License ID/SPDX Identifier)

A valid SPDX short-form license identifier, as described in the Add License-Expression field section of this PEP; briefly, this includes all valid SPDX identifiers and the LicenseRef-Public-Domain and LicenseRef-Proprietary strings. Examples: MIT, GPL-3.0-only

Project (Sub: Project Source Tree, Installed Project)

(See PyPUG) Here, a project source tree refers to the on-disk format of a project used for development, while an installed project is the form a project takes once installed from a distribution, as specified by PyPA.

Project Source Metadata (Sub: Project Table Metadata, Key, Subkey)

Core metadata defined by the package author in the project source tree, as top-level keys in the [project] table of a pyproject.toml file, in the [metadata] table of setup.cfg, or the equivalent for other build tools.

The Project Table Metadata, or pyproject.toml [project] metadata, refers specifically to the former, as defined by the PyPA Declaring Project Metadata specification and originally specified in PEP 621. A Project Table Key, or an unqualified key refers specifically to a top-level [project] key (notably, distinct from a core metadata field), while a subkey refers to a second-level key in a table-valued [project] key.

Root License Directory (Short: License Directory)

The directory under which license files are stored in a project/distribution and the root directory that their paths, as recorded under the License-File core metadata fields, are relative to. Defined here to be the project root directory for source trees and source distributions, and a subdirectory named licenses of the directory containing the core metadata (i.e., the .dist-info/licenses directory) for built distributions and installed projects.

Tool (Sub: Packaging Tool, Build Tool, Install Tool, Publishing Tool)

A program, script or service executed by the user or automatically that seeks to conform to the specification defined in this PEP.

A packaging tool refers to a tool used to build, publish, install, or otherwise directly interact with Python packages.

A build tool is a packaging tool used to generate a source or built distribution from a project source tree or sdist, when directly invoked as such (as opposed to by end-user-facing install tools). Examples: Wheel project, PEP 517 backends via build or other package-developer-facing frontends, calling setup.py directly.

An install tool is a packaging tool used to install a source or built distribution in a target environment. Examples include the PyPA pip and installer projects.

A publishing tool is a packaging tool used to upload distribution archives to a package index, such as Twine for PyPI.

Wheel (Short: wheel, Rel: wheel format, Wheel project)

Here, wheel, the standard built distribution format introduced in PEP 427 and specified by the PyPA, will be referred to in lowercase, while the Wheel project, its reference implementation, will be referred to as such with Wheel in Title Case.

Specification

The changes necessary to implement the improved license handling outlined in this PEP include those in both distribution package metadata, as defined in the core metadata specification, and author-provided project source metadata, as defined in the project source metadata specification (and originally introduced in PEP 621).

Further, minor additions to the source distribution (sdist), built distribution (wheel) and installed project specifications will help document and clarify the already allowed, now formally standardized behavior in these respects. Finally, guidance is established for tools handling and converting legacy license metadata to license expressions, to ensure the results are consistent, correct and unambiguous.

Note that the guidance on errors and warnings is for tools’ default behavior; they MAY operate more strictly if users explicitly configure them to do so, such as by a CLI flag or a configuration option.

license-files.globs = ["LICEN[CS]E*", "COPYING*", "NOTICE*", "AUTHORS*"]

Backwards Compatibility

Adding a new, dedicated License-Expression core metadata field and a top-level string value for the license key reserved for this purpose in the pyproject.toml [project] table unambiguously signals support for the specification in this PEP. This avoids the risk of new tooling misinterpreting a license expression as a free-form license description or vice versa, and raises an error if and only if the user affirmatively upgrades to the latest metadata version and adds the new field/key.

The legacy License core metadata field and the license key table subkeys (text and file) in the pyproject.toml [project] table will be deprecated along with the license classifiers, retaining backwards compatibility while gently preparing users for their future removal. Such a removal would follow a suitable transition period, and be left to a future PEP and a new version of the core metadata specification.

Formally specifying the new License-File core metadata field and the inclusion of the listed files in the distribution merely codifies and refines the existing practices in popular packaging tools, including the Wheel and Setuptools projects, and is designed to be largely backwards-compatible with their existing use of that field. Likewise, the new license-files key in the [project] table of pyproject.toml standardizes statically specifying the files to include, as well as the default behavior, and allows other tools to make use of them, while only having an effect once users and tools expressly adopt it.

Due to requiring license files not be flattened into .dist-info and specifying that they should be placed in a dedicated licenses subdir, wheels produced following this change will have differently-located licenses relative to those produced via the previous unspecified, installer-specific behavior, but as until this PEP there was no way of discovering these files or accessing them programmatically, and this will be further discriminated by a new metadata version, there aren’t any foreseen mechanism for this to pose a practical issue.

Furthermore, this resolves existing compatibility issues with the current ad hoc behavior, namely license files being silently clobbered if they have the same names as others at different paths, unknowingly rendering the wheel undistributable, and conflicting with the names of other metadata files in the same directory. Formally specifying otherwise would in fact block full forward compatibility with additional standard or installer-specified files and directories added to .dist-info, as they too could conflict with the names of existing licenses.

While minor additions will be made to the source distribution (sdist), built distribution (wheel) and installed project specifications, all of these are merely documenting, clarifying and formally specifying behaviors explicitly allowed under their current respective specifications, and already implemented in practice, and gating them behind the explicit presence of both the new metadata versions and the new fields. In particular, sdists may contain arbitrary files following the project source tree layout, and formally mentioning that these must include the license files listed in the metadata merely documents and codifies existing Setuptools practice. Likewise, arbitrary installer-specific files are allowed in the .dist-info directory of wheels and copied to installed projects, and again this PEP just formally clarifies and standardizes what is already being done.

Finally, while this PEP does propose PyPI implement validation of the new License-Expression and License-File fields, this has no effect on existing packages, nor any effect on any new distributions uploaded unless they explicitly choose to opt in to using these new fields while not following the requirements in the specification. Therefore, this does not have a backward compatibility impact, and in fact ensures forward compatibility with any future changes by ensuring all distributions uploaded to PyPI with the new fields are valid and conform to the specification.

Security Implications

This PEP has no foreseen security implications: the License-Expression field is a plain string and the License-File fields are file paths. Neither introduces any known new security concerns.

How to Teach This

The simple cases are simple: a single license identifier is a valid license expression, and a large majority of packages use a single license.

The plan to teach users of packaging tools how to express their package’s license with a valid license expression is to have tools issue informative messages when they detect invalid license expressions, or when the deprecated License field or license classifiers are used.

An immediate, descriptive error message if an invalid License-Expression is used will help users understand they need to use SPDX identifiers in this field, and catch them if they make a mistake. For authors still using the now-deprecated, less precise and more redundant License field or license classifiers, packaging tools will warn them and inform them of the modern replacement, License-Expression. Finally, for users who may have forgotten or not be aware they need to do so, publishing tools will gently guide them toward including license and license-files in their project source metadata.

Tools may also help with the conversion and suggest a license expression in many, if not most common cases:

• The appendix Appendix: Mapping License Classifiers to SPDX Identifiers provides tool authors with recommendation on how to suggest a license expression produced from legacy classifiers.
• Tools may also be able to infer and suggest how to update an existing License value in project source metadata and convert that to a license expression, as also specified in this PEP. For instance, a tool may suggest converting a value of MIT in the license.text key in [project] (or the equivalent in tool-specific formats) to a top-level string value of the license key (or equivalent). Likewise, a tool could suggest converting from a License of Apache2 (which is not a valid license expression as defined in this PEP) to a License-Expression of Apache-2.0 (the equivalent valid license expression using an SPDX license identifier).

Reference Implementation

Tools will need to support parsing and validating license expressions in the License-Expression field.

The license-expression library is a reference Python implementation that handles license expressions including parsing, formatting and validation, using flexible lists of license symbols (including SPDX license IDs and any extra identifiers included here). It is licensed under Apache-2.0 and is already used in several projects, including the SPDX Python Tools, the ScanCode toolkit and the Free Software Foundation Europe (FSFE) REUSE project.

Rejected Ideas

Many alternative ideas were proposed and after a careful consideration, rejected. The exhaustive list including the rationale for rejecting can be found in a separate page.

Open Issues

Appendices

A list of auxilliary documents is provided:

• Detailed Licensing Examples,
• User Scenarios,
• License Documentation in Python and Other Projects,
• Mapping License Classifiers to SPDX Identifiers,
• Rejected Ideas in detail.

Acknowledgments

• Alyssa Coghlan
• Kevin P. Fleming
• Pradyun Gedam
• Oleg Grenrus
• Dustin Ingram
• Chris Jerdonek
• Cyril Roelandt
• Luis Villa

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.