Background

Python’s Stable ABI, as defined in PEP 384 and PEP 652, provides a way to compile extension modules that can be loaded on multiple minor versions of the CPython interpreter. Several projects use this to limit the number of wheels (binary artefacts) that need to be built and distributed for each release, and/or to make it easier to test with pre-release versions of Python.

With free-threading builds (PEP 703) being on track to eventually become the default (PEP 779), we need a way to make the Stable ABI available to those builds.

To build against the Stable ABI, the extension must use a Limited API, that is, only a subset of the functions, structures, etc. that CPython exposes. Both the Limited API and the Stable ABI are versioned, and building against Stable ABI 3.X requires using only Limited API 3.X, and yields an extension that is ABI-compatible with CPython 3.X and any later version (though bugs in CPython sometimes cause incompatibilities in practice).

The Limited API is not “stable”: newer versions may remove API that were a part of older versions. This PEP proposes the most significant such removal to date.

Opaque PyObject

Version 3.15 of the Limited API will:

• make the following structures opaque (or in C terminology, incomplete types):
  • PyObject
  • PyVarObject
  • PyModuleDef_Base
  • PyModuleDef
• no longer include the following macros:
  • PyObject_HEAD
  • _PyObject_EXTRA_INIT
  • PyObject_HEAD_INIT
  • PyObject_VAR_HEAD
  • Py_SET_TYPE()
• export the following as functions in the ABI, rather than macros:
  • Py_SIZE()
  • Py_SET_SIZE()

New Export Hook (PEP 793)

Implementation of this PEP requires PEP 793 (PyModExport): A new entry point for C extension modules), which was accepted for Python 3.15.

Since existing ways of defining modules use API that this PEP removes (namely, PyModuleDef), extensions will need to migrate to PEP 793’s new “export hook” when switching to Limited API 3.15.

Runtime ABI checks

Users – or rather the tools they use for building and installing extensions – will continue to be responsible for not putting incompatible extensions on Python’s import paths. This decision makes sense since tools typically have much richer metadata than what CPython can check. Typically, build tools and installers use PyPA packaging metadata and platform compatibility tags to communicate compatibility details, but other models are possible.

However, CPython will add a line of defense against outdated or misconfigured tools, or human mistakes, in the form of a new module slot containing basic ABI information. This information will be checked when a module is loaded, and incompatible extensions will be rejected. The specifics are left to the C API working group (see issue 72).

This slot will become mandatory with the new export hook added in PEP 793. (That PEP currently says “there are no required slots”; it will be updated.)

The abi3t wheel tag

Wheels that use a stable ABI compatible with free-threading CPython builds should use a new ABI tag: abi3t. The name is chosen to reflect the fact that this ABI is similar to abi3, with limitations necessary to support free-threading (which uses the letter t in existing, version-specific ABI tags like cp314t).

Package installers should treat this tag as completely separate from abi3. They should allow abi3t-tagged wheels for free-threaded builds wherever they currently allow abi3-tagged ones for (orherwise equal) non-free-threaded builds.

Build tools should generate abi3.abi3t instead of abi3 when the Python tag is cp315 and above (or equivalently: when setting Py_LIMITED_API to 3.15 (0x030f0000) or above). abi3.abi3t is a compressed tag set that signals compatibility with both abi3 and abi3t.

The abi3t tag can be used in extensions compatible with earlier versions of free-threaded Python. For example, an extension compatible with GIL-enabled and free-threaded builds of 3.14, 3.15, and higher versions would be tagged cpy314-abi3.abi3t. This PEP does not propose an official way to build such extensions, but since a mechanism for that can be added later (see a Rejected idea), installers should be ready to accept the tag.

New API

Implementing this PEP will make it possible to build extensions that can be successfully loaded on free-threaded Python, but not necessarily ones that are thread-safe without a GIL.

Limited API to allow thread-safety without a GIL – presumably PyMutex, PyCriticalSection, and similar – will be added via the C API working group, or in a follow-up PEP.

Compatibility Overview

The following table summarizes compatibility of wheel tags with CPython interpreters. “GIL” stands for GIL-enabled interpreter; “FT” stands for a free-threaded one.

(*): Wheels with these tags cannot be built; see table below

The following table summarizes which wheel tag should be used for an extension built with a given interpreter and Py_LIMITED_API macro:

Values in the Note column:

• existing: The wheel tag is currently in use
• continued: The wheel tag continues the existing scheme
• discontinued: The wheel tag continues the existing scheme, but it will be discouraged. Older tools may still generate it. Installers will continue to accept it, but only for GIL-enabled builds, even though the wheel would be compatible with free-threaded ones.
• new: Proposed in this PEP.
• reserved: A mechanism to build a matching extension is not proposed in this PEP, but may be added in the future. Installers should be prepared to handle the tag.
• out of spec: Should not be used as-is: extensions should be tagged abi3.abi3t rather than only abi3. The entry is included for installers that decompose compressed tag sets.

Add an alternative stable ABI for free-threading

It would be possible to:

• Add a new stable ABI (“abi3t”) specifically for free-threading, which would be incompatible with the existing abi3. Extensions would need no code changes to target abi3t and builds would be compatible with free-threaded CPython (3.14 and above).
• Define an additional macro (“Py_OPAQUE_PYOBJECT”), which would make PyObject opaque as in this PEP. Extensions would need code changes as in this PEP, and as in this PEP, compiled extensions (“abi3.abi3t”) would be compatible with all builds of CPython 3.15+.

This scheme was rejected as too complex. It would also make the free-threading memory layout of PyObject part of the stable ABI, preventing future adjustments.

Shims for compatibility with CPython 3.14

It’s possible to build a cp314-abi3.abi3t extension – one compatible with 3.14 (both free-threaded build and default). There are several challenges around this:

• making it convenient and safe for general extensions
• testing it (as CPython’s test suite doesn’t involve other CPython versions than the one being tested)

So, providing a mechanism to build such extensions is best suited to an external project (for example, one like pythoncapi-compat). It’s out of scope for CPython’s C API, and this PEP.

To sketch how such a mechanism could work:

The main issue that prevents compatibility with Python 3.14 is that with opaque PyObject and PyModuleDef, it is not feasible to initialize an extension module. The solution, PEP 793, is only being added in Python 3.15.

It is possible to work around this using the fact that the 3.14 ABIs (both free-threading and GIL-enabled) are “frozen”, so it is possible for an extension to query the running interpreter, and for 3.14, use a struct definition corresponding to the detected build’s PyModuleDef.

Using the Python version wheel tag to determine compatibility

A previous version of this PEP avoided adding a new wheel tag (abi3t), and specified that wheels tagged abi3 would be compatible with free-threading if the Python tag is cp315 or higher.

Such a scheme would work for this PEP, but it cannot express that an extension is compatible with both GIL-enabled and free-threaded builds of CPython 3.14 or lower. Adding a new explicit tag means that if we allow building such wheels in the future, packaging tools should not need additional changes to support them. They would be tagged cp314-abi3.abi3t.

Adding an abi4 wheel tag

Instead of abi3t, we could “bump the version” and use abi4 instead as the wheel ABI tag. In the wheel tag, the difference is largely cosmetic.

However, one thing this PEP does not propose is changing the filename tag: extensions will be named with the extensions like .abi3.so. Changing this while keeping compatibility with GIL-enabled builds would be an unnecessary technical change.

Using abi3.abi4 in wheel tags but only .abi3 in filenames would look more inconsistent than abi3.abi3t and .abi3.

If we add abi4 tag, the Py_LIMITED_API value would either need to:

• change to start with 4 to match abi4, but no longer correspond to PY_VERSION_HEX (making it harder to generate and check), or
• not change, making it inconsistent with abi4.

Adding abi3t is a smaller change than adding abi4, making it work better as a transitional state before larger changes like PEP 809’s abi2026.