Background & Summary

Python’s Stable ABI (abi3 for short), as defined in PEP 384 and PEP 652, provides a way to compile extension modules that can be loaded on multiple minor versions of the CPython interpreter. Several projects use this to limit the number of wheels (binary artefacts) that need to be built and distributed for each release, and/or to make it easier to test with pre-release versions of Python.

With free-threading builds (PEP 703) being on track to eventually become the default (PEP 779), we need a way to make a Stable ABI available to those builds.

The current Stable ABI is versioned, and an extension built for Stable ABI 3.X is ABI-compatible with CPython 3.X and any later version (though bugs in CPython sometimes cause incompatibilities in practice).

However, this forward compatibility is only guaranteed for a subset of the API that CPython exposes (functions, structures, etc.). Extensions that target the Stable ABI must limit themselves to this subset, called the Limited API. When the “opt-in” preprocessor macro Py_LIMITED_API is defined, CPython headers will expose the Limited API only

This PEP proposes a stable ABI for free-threading builds (abi3t for short), which includes additional API limitations needed to compile extensions compatible with both GIL-enabled and free-threaded builds of CPython 3.15+, a corresponding macro to opt in to these limitations, and naming/tagging schemes that extensions should use to signal such compatibility.

Ecosystem maintainers want decreased maintenance burden

A major advantage of the limited API and stable ABI wheels is that new Python versions are supported on the day of release. Without stable ABI wheels, maintainers are left with a choice between closely following CPython releases and producing wheels during CPython beta periods or dealing with inevitable user requests for support for new CPython versions.

Bindings generators

Both moocore and cryptography use bindings generators to interface with the C API. Cryptography uses PyO3 and CFFI while moocore uses only CFFI. Both CFFI and PyO3 already handle all the details of abstracting over the C API to enable different build configurations and there is no need to laboriously port extension types to use APIs that are only available on one build or another.

Using bindings generators will enable these projects to quickly adopt the new stable ABI. Initial testing using the experimental _Py_OPAQUE_PYOBJECT flag defined in CPython’s main branch, indicates that PyO3, CFFI, and Cython will all work with PEP 803 using packaging tools that have been patched to account for an abi3.abi3t tag.

PyO3 maintainer David Hewitt said in support of this proposal:

PyO3 greatly benefits from having a stable ABI - one of the biggest challenges for the framework is the need to abstract over a wide range of Python / OS / CPU / environment combinations. We also offer the possibility to build with the stable ABI for each of these environments (targeting a given minimum Python version’s stable ABI). The goal is always that all functionality PyO3 offers works the same on all these combinations (sometimes with a Python version floor to access certain features). We currently support Python 3.7+. All functionality added to the stable ABI is a very welcome promise that PyO3 will not need to introduce further conditional code to support a given feature. In the long run this makes it possible for PyO3 to simplify code paths as support for older Python versions is dropped, helping to keep maintenance burden under control.

When asked to comment about this proposal, Cython maintainer David Woods said:

Cython doesn’t have huge problems with the number of wheels we distribute because ultimately it works fine as pure-Python. We do distribute wheels for a few of the smaller platforms as Stable ABI wheels but that’s more “dogfooding” than because we actually need to. So I’m adding this in anticipation that other people will find it useful rather than because I will.

I do remain slightly concerned that the performance trade-offs for this will turn out to be too much for many Cython users (it’s possible that the trade-off may be different for other binding tools). That’s not a huge disaster since we’re not getting rid of the regular compilation mode so people are free to pick their own personal trade-offs.

It should be noted that this PEP leaves some questions/work open. Wenzel Jakob – maintainer of nanobind, a C++ binding generator – noted a need for additional API that is left out of scope of this PEP:

It’s not clear to me how I can get a pointer to the N-th data entry of [a PyVarObject-derived object].

If that can be resolved then nanobind should be able to adopt this new abi3t compilation target.

Separate ABI

The new ABI (abi3t) will conceptually be treated as separate from the existing Stable ABI (abi3) – even though all extensions compatible with abi3t will, in practice, also be compatible with abi3.

(In more precise wording: abi3t’s set of allowed APIs will be a subset of abi3’s; abi3t’s set of compatible interpreters will be a superset of abi3’s. That makes one’s head spin, which is part of the reason to keep them separate.)

Extensions should be explicitly compiled for both abi3t and abi3, and should explicitly signal that they support both at once – via packaging wheel tags (abi3.abi3t) and a runtime ABI check (Py_mod_abi).

This explicitness has several advantages over having abi3t support imply abi3 support:

• The tags clearly show whether an extension is compatible with GIL-enabled builds, and whether the existing backwards compatibility guarantees of abi3 apply.
• Implementation-wise, the set of tags a given free-threaded interpreter supports (as returned from the packaging function packaging.tags.sys_tags) It will be the same size as for a corresponding GIL-enabled build.
• It allows the ABIs to diverge slightly in the future – keeping both at once as the preferred compilation target, but allowing abi3t-only extensions for special cases.

One practical exception to keeping the ABIs conceptually separate is discussed in the Filename tag section.

See these Rejected Ideas sections for more on the alternatives:

• Making abi3t compatible with abi3
• Single new ABI: make PyObject opaque in Stable ABI 3.15

No backwards compatibility now

CPython headers will not allow compiling for abi3t for CPython 3.14 and earlier. Projects that need this can build separate extensions specifically for the 3.14 free-threaded interpreter, and for older abi3.

However, it is technically possible to build an extension compatible with both free-threaded and GIL-enabled builds of CPython 3.14+. To enable experiments in this area, we recommend that package installation tools are prepared for such extensions. See a rejected idea for more details.

Source changes are necessary in extensions

abi3t will require extension authors to make significant changes to their code.

Projects that cannot do this (yet) can continue using abi3, and compile the same source for specific versions of free-threaded builds. (Note that the APIs removed in api3t still are usable when compiling for a specific version, including 3.15t.)

See a Rejected Ideas sections for an alternative: Fully separate ABIs, keeping source compatibility

Tag name

The tag abi3t is chosen to reflect the fact that this ABI is similar to abi3, with minimal changes necessary to support free-threading (which uses the letter t in existing, version-specific ABI tags like cp314t).

See a Rejected Ideas sections for an alternative: Naming this abi4

Filename tag

On systems that use the abi3 tag in filenames, a new filename tag (abi3t) is added so that older stable ABI extensions (name.abi3.so) can be installed in the same directory as ones that support Stable ABI for free-threaded Python (name.abi3t.so).

There can only be one ABI tag in a filename (there is no concept of “compressed tag sets like in wheel tags), so extensions that are compatible with both ABIs at once need to use one of the tags – the new one (abi3t), as the existing one has existing meaning.

See Rejected Ideas sections for alternatives:

• abi3+abi3t filename tag
• No filename tag (bare .so)

Knob name

This PEP specifies that the C preprocessor macro Py_TARGET_ABI3T will enable compiling for abi3t (that is, practically: it will make Python.h only expose forward-compatible definitions).

The corresponding “knob” for abi3 is named Py_LIMITED_API. This name is problematic:

• It describes what the macro’s historical internal effect (limiting which definitions are exposed), but not the intended benefit (forward compatibility).
• It is increasingly a misnomer: for API like Py_TYPE, it selects a forward-compatible implementation (DLL function call rather than inline pointer deference) rather than limiting the API.
• The pair of terms Stable ABI and Limited API is technically accurate, but quite confusing. Avoiding the term Limited API, and talking about “constraints necessary for targeting a given ABI”, tends to be clearer.

The proposed macro name (Py_TARGET_ABI3T) emphasizes abi3t as a compilation target, with API limitations as its implicit price – and forward compatibility as the implicit benefit.

As for Py_LIMITED_API, this PEP proposes no change, which means keeping it for abi3. abi3 is expected to eventually become irrelevant if free-threaded builds replace the GIL-enabled ones (see the PEP 703 acceptance notice for the tentative plan). At that point, Py_LIMITED_API will likely remain user-visible, but as an implementation detail.

See a Rejected Ideas sections for an alternative – reusing an existing “knob”: Reusing Py_GIL_DISABLED to enable the new ABI

Stable ABI for free-threaded builds

Python will introduce a new stable ABI, called stable ABI for free-threading builds, or abi3t for short. As with the current Stable ABI (abi3), abi3t will be versioned using major (3) and minor versions of Python interpreter. Extensions built for abi3t 3.x will be compatible with free-threading builds of CPython 3.x and above.

This mirrors the compatibility promise for the existing Stable ABI, abi3, which was defined PEP 384 and modified in PEP 703: Extensions built for abi3 3.x will be compatible with GIL-enabled builds of CPython 3.x and above.

To build a C/C++ extension for abi3t, the extension will need to limit itself to only use API for which we can promise long-term support. This limited API for free-threaded builds will be a subset of the 3.15 Limited API.

Any extension compiled for abi3t will, in practice, be compatible with abi3 as well. However, we recommend that users and tools explicitly compile for both at the same time, and signal this explicitly. (In the PyPA packaging ecosystem, this signaling means using the wheel tag abi3.abi3t as detailed below).

Choosing the target ABI

Users of the C API – or build tools acting on their behalf, configured by tool-specific UI – will select the target ABI using the following macros:

• Py_LIMITED_API=<version> (existing): Compile for abi3 of the given version.
• Py_TARGET_ABI3T=<version> (proposed here): Compile for abi3t of the given version.

For ease of use and implementation simplicity, respectively, Python.h will set the configuration macros automatically in the following situations:

• If Py_LIMITED_API=v and Py_GIL_DISABLED is set, then Py_TARGET_ABI3T will be defined as v by default. (This allows choosing abi3t by defining the pre-existing macro and compiling with free-threaded CPython headers. Note that in CPython 3.14, this case results in a compile-time error.)
• If Py_TARGET_ABI3T=v is set, CPython may define or redefine Py_LIMITED_API as v. (This means that CPython can continue to use the Py_LIMITED_API macro internally to select which APIs are available.)

Opaque PyObject

abi3t will initially have a single difference from abi3: the PyObject structure and APIs that depend on it are not part of abi3t.

Specifically, when building for abi3t, the CPython headers will:

• make the following structures opaque (or in C terminology, incomplete types):
  • PyObject
  • PyVarObject
  • PyModuleDef_Base
  • PyModuleDef
• no longer include the following macros:
  • PyObject_HEAD
  • _PyObject_EXTRA_INIT
  • PyObject_HEAD_INIT
  • PyObject_VAR_HEAD
  • Py_SET_TYPE()

In both the regular stable ABI (abi3 3.15+) and the new abi3t, the following will be exported functions (exposed in the ABI) rather than macros:

• Py_SIZE()
• Py_SET_SIZE()
• Py_IS_TYPE()

Runtime ABI checks

Users – or rather build/install tools acting on users’ behalf – will continue to be responsible for not putting incompatible extensions on Python’s import paths. This decision makes sense since tools typically have much richer metadata than what CPython can check. Typically, build tools and installers use PyPA packaging metadata and platform compatibility tags to communicate compatibility details, but other models are possible.

However, CPython will add a line of defense against outdated or misconfigured tools, or human mistakes, in the form of a new module slot, Py_mod_abi, containing basic ABI information. This information will be checked when a module is loaded, and incompatible extensions will be rejected. The specifics are left to the C API working group (see capi-workgroup issue 72).

This slot will become mandatory with the new export hook added in PEP 793. (That PEP currently says “there are no required slots”; it will be updated.)

The abi3t wheel and filename tags

Wheels with extension modules compiled for Stable ABI for Free-Threaded Python should use a new ABI tag: abi3t.

On systems where filenames of Stable ABI extensions end with .abi3.so, extensions that support free-threading should instead use abi3t.so. This includes extensions compatible with both abi3 and abi3t.

On these systems, all builds of CPython – GIL-enabled and free-threaded – will load extensions with the abi3t tag. Free-threaded builds will – unlike in 3.14 – not load extensions with the abi3 tag. If files are present with both tags, GIL-enabled builds will prefer “their” *.abi3.so over *.abi3t.so.

Put another way, importlib.machinery.EXTENSION_SUFFIXES will be (for x86_64-linux-gnu builds of CPython):

• python3.15: ['.cpython-315-x86_64-linux-gnu.so', '.abi3.so', '.abi3t.so', '.so']
• python3.15t: ['.cpython-315-x86_64-linux-gnu.so', '.abi3t.so', '.so']

Making GIL-enabled builds load .abi3t.so files is purely a practical choice: it this one case we break the conceptual purity of abi3 and abi3t being separate ABIs.

New API

Implementing this PEP will make it possible to build extensions that can be successfully loaded on free-threaded Python, but not necessarily ones that are thread-safe without a GIL.

Limited API to allow thread-safety without a GIL – presumably PyCriticalSection and similar – will be added via the C API working group, or in a follow-up PEP.

Compatibility Overview

The following table summarizes compatibility of wheel tags with CPython interpreters. “GIL” stands for GIL-enabled interpreter; “FT” stands for a free-threaded one.

(*): Wheels with these tags cannot be built; see table below

The following table summarizes which wheel tag should be used for an extension built with the given interpreter and defined macros:

In the “Compile on” column, FT means that the Py_GIL_DISABLED macro must be defined – either explicitly or, on non-Windows platforms, by including CPython headers configured with --disable-gil. GIL means that Py_GIL_DISABLED must not be defined. Rows without this note apply to both cases.

In the Py_LIMITED_API and Py_TARGET_ABI3T, a dash means the macro must not be defined; a version means the macro must be set to the corresponding integer in Py_PACK_VERSION() format.

Values in the Note column:

• existing: The wheel tag is currently in use
• continued: The wheel tag continues the existing scheme
• new: Proposed in this PEP.
• reserved: A mechanism to build a matching extension is not proposed in this PEP, but may be added in the future. Installers should be prepared to handle the tag.

Single new ABI: make PyObject opaque in Stable ABI 3.15

It would be possible to make PyObject struct opaque in Stable ABI rather than introduce a new variant of the Stable ABI.

This would mean that extension authors would need to adapt their code to the new limitations, or abandon Stable ABI altogether, in order to use any C API introduced in Python 3.15.

It would also not fully remove the case for a new wheel tag (abi3t), which would be required to express that an extension is compatible with both GIL-enabled and free-threaded builds of CPython 3.14 or lower.

In the PEP discussion, the ability to build for the GIL-only Stable ABI with no source changes was deemed to be worth an extra configuration macro (now called Py_TARGET_ABI3T).

Shims for compatibility with CPython 3.14

It’s possible to build a cp314-abi3.abi3t extension – one compatible with 3.14 (both free-threaded build and default). There are several challenges around this:

• making it convenient and safe for general extensions
• testing it (as CPython’s test suite doesn’t involve other CPython versions than the one being tested)

So, providing a mechanism to build such extensions is best suited to an external project (for example, one like pythoncapi-compat). It’s out of scope for CPython’s C API, and this PEP.

To sketch how such a mechanism could work:

The main issue that prevents compatibility with Python 3.14 is that with opaque PyObject and PyModuleDef, it is not feasible to initialize an extension module. The solution, PEP 793, is only being added in Python 3.15.

It is possible to work around this using the fact that the 3.14 ABIs (both free-threading and GIL-enabled) are “frozen”, so it is possible for an extension to query the running interpreter, and for 3.14, use a struct definition corresponding to the detected build’s PyModuleDef.

Making abi3t compatible with abi3

It would be possible to teach packaging tools that abi3t is a “subset” of abi3, that is, all GIL-enabled interpreters are guaranteed to be compatible with abi3t-tagged builds. This would make the abi3.abi3t compressed tag set equivalent to abi3t, and thus redundant. However, tools would still need to output the compressed tag set to support “older installers”, which do not consider abi3t compatible with GIL-enabled builds.

Here, “older installers” include ones that use or vendor an version of the packaging library that wasn’t updated for abi3t. (The packaging library is what Python-based installers typically use to implement wheel tag matching.)

Beyond installers, the abi3.abi3t tag allow mechanisms like the ABI filter in PyPI file list (e.g. on https://pypi.org/project/cryptography/#files) to match abi3 without special-casing (assuming compressed tags are handled according to standard). Humans wondering about compatibility with abi3 also get a more explicit signal.

Less importantly, merging the ABIs would also remove an “escape hatch” of possibly making abi3 and abi3t diverge in the future.

Naming this abi4

Instead of abi3t, we could “bump the version” and use abi4 instead. The difference is largely cosmetic.

If we added an abi4 tag, the value of the opt-in macro (Py_TARGET_ABI4 or Py_LIMITED_API or some such) would either need to:

• change to start with 4 to match abi4, but no longer correspond to PY_VERSION_HEX (making it harder to generate and check), or
• not change, making it inconsistent with abi4.

Adding abi3t is a smaller change than adding abi4, making it work better as a transitional state before larger changes like PEP 809’s abi2026.

abi3+abi3t filename tag

Filename ABI tags (as introduced in PEP 3149) allow extensions for several ABIs to co-exist in a directory.

Per this PEP, extensions that are compatible with both abi3 and abi3t will use a compressed tag set (abi3.abi3t) in wheel metadata, but not in filenames (.abi3.so/.abi3t.so). We could add a dedicated tag for the combination – for example, .abi3+abi3t.so.

But, there would be no need for .abi3+abi3t.so extensions to co-exist with .abi3t.so ones: free-threaded interpreters would always pick .abi3t.so, so the extension for GIL-enabled interpreters could just as well use .abi3.so. The only benefit would be clearer naming when an abi3.abi3t extension is not installed together with its abi3-only equivalent.

Here, clearer naming is not worth the complexity. We make the practical choice to make .abi3t.so mean “abi3+abi3t”, that is, “loadable by all builds”. This works for (discouraged) abi3t-only extensions: on a GIL-enabled interpreter, these will fail the mandatory runtime ABI check or, in the unlikely future where abi3t & abi3 diverge, possibly fail to load due to a missing linker symbol.

Conceptually, filename tags do not “describe” or “name” an extension’s ABI. The current .abi3 tag is already too weak for this, as it lacks a version.

No filename tag (bare .so)

It would be possible to drop the filename ABI tag altogether, and use .so instead of .abi3t.so. The practical meaning of these two tags is very close: .so is loadable by any build of CPython; .abi3t.so will be loadable by any CPython 3.15 or above – but the Stable ABI filename tag already lacks version information.

They are different semantically, though. Bare .so means “don’t care”; an .abi3t.so extension is intentionally compatible with the new ABI.

Reusing Py_GIL_DISABLED to enable the new ABI

It would be possible to select abi3t (rather than abi3) when the Py_GIL_DISABLED macro is defined together with Py_LIMITED_API.

This would require annoying fiddling with build flags, and make it impossible to explicitly target both abi3 and abi3t at the same time.

Fully separate ABIs, keeping source compatibility

It would be possible to make abi3 and abi3t fully separate and incompatible. This would allow all current extensions to stay source-compatible with abi3t: the PyObject struct could stay exposed, with each ABI defining a different set of private fields, as in the version-specific CPython ABI.

However, exposed PyObject struct has been noted as one of the main shortcomings of the existing Stable ABI. It hindered or prevented optimizations and features such as immortalization and free-threading itself.

Exposing PyObject would mean repeating this mistake, “freezing” its current free-threaded definition, and requiring yet another variant of stable ABI if/when changes are needed.