PEP 774 – Removing the LLVM requirement for JIT builds
- Author:
- Savannah Ostrowski <savannah at python.org>
- Discussions-To:
- Discourse thread
- Status:
- Draft
- Type:
- Standards Track
- Created:
- 27-Jan-2025
- Python-Version:
- 3.14
Table of Contents
Abstract
Since Python 3.13, CPython has been able to be configured and built with an
experimental just-in-time (JIT) compiler via the --enable-experimental-jit
flag on Linux and Mac and --experimental-jit
on Windows. To build CPython with
the JIT enabled, users are required to have LLVM installed on their machine
(initially, with LLVM 16 but more recently, with LLVM 19). LLVM is responsible
for generating stencils that are essential to our copy-and-patch JIT (see PEP 744).
These stencils are predefined, architecture-specific templates that are used
to generate machine code at runtime.
This PEP proposes removing the LLVM build-time dependency for JIT-enabled builds by hosting the generated stencils in the CPython repository. This approach allows us to leverage the checked-in stencils for supported platforms at build time, simplifying the contributor experience and address concerns raised at the Python Core Developer Sprint in September 2024. That said, there is a clear tradeoff to consider, as improved developer experience does come at the cost of increased repository size.
It is important to note that this PEP is not a proposal to accept or reject the JIT itself but rather to determine whether the build-time dependency on LLVM is acceptable for JIT builds moving forward. If this PEP is rejected, we will proceed with the status quo, retaining the LLVM build-time requirement. While this dependency has served the JIT development process effectively thus far, it introduces setup complexity and additional challenges that this PEP seeks to alleviate.
Motivation
At the Python Core Developer Sprint that took place in September 2024, there was discussion about the next steps for the JIT - a related discussion also took place on GitHub. As part of that discussion, there was also a clear appetite for removing the LLVM requirement for JIT builds in preparation for shipping the JIT off by default in 3.14. The consensus at the sprint was that it would be sufficient to provide pre-generated stencils for non-debug builds for Tier 1 platforms and that checking these files into the CPython repo would be adequate for the limited number of platforms (though more options have been explored; see Rejected Ideas).
Currently, building CPython with the JIT requires LLVM as a build-time dependency. Despite not being exposed to end users, this dependency is suboptimal. Requiring LLVM adds a setup burden for developers and those who wish to build CPython with the JIT enabled. Depending on the operating system, the version of LLVM shipped with the OS may differ from that required by our JIT builds, which introduces additional complexity to troubleshoot and resolve. With few core developers currently contributing to and maintaining the JIT, we also want to make sure that the friction to work on JIT-related code is minimized as much as possible.
With the proposed approach, hosting pre-compiled stencils for supported architectures can be generated in advance, stored in a central location, and automatically used during builds. This approach ensures reproducible builds, making the JIT a more stable and sustainable part of CPython’s future.
Rationale
This PEP proposes checking JIT stencils directly into the CPython repo as the best path forward for eliminating our build-time dependency on LLVM.
This approach:
- Provides the best end-to-end experience for those looking to build CPython with the JIT
- Lessens the barrier to entry for those looking to contribute to the JIT
- Ensures builds remain reproducible and consistent across platforms without relying on external infrastructure or download mechanisms
- Eliminates variability introduced by network conditions or potential discrepancies between hosted files and the CPython repository state, and
- Subjects stencils to the same review processes we have for all other JIT-related code
However, this approach does result in a slight increase in overall repository size. Comparing repo growth on commits over the past 90 days, the difference between the actual commits and the same commits with stencils added amounts to a difference of 0.03 MB per stencil file. This is a small increase in the context of the overall repository size, which has grown by 2.55 MB in the same time period. For six stencil files, this amounts to an upper bound of 0.18 MB. The current total size of the stencil files for all six platforms is 7.2 MB.
These stencils could become larger in the future with changes to register allocation, which would introduce 5-6 variants per instruction in each stencil file (5-6x larger). However, if we ended up going this route, there are additional modifications we could make to stencil files that could help counteract this size increase (e.g., stripping comments, minimizing the stencils).
Specification
This specification outlines the proposed changes to remove the build-time dependency on LLVM and the contributor experience if this PEP is accepted.
Repository changes
The CPython repository would now host the pre-compiled JIT stencils in a new
subdirectory in Tools/jit
called stencils/
. At present, the JIT is tested
and built for six platforms, so to start, we’d check in six stencil files. In
the future, we may check in additional stencil files if support for additional
platforms is desired or relevant.
cpython/
Tools/
jit/
stencils/
aarch64-apple-darwin.h
aarch64-unknown-linux-gnu.h
i686-pc-windows-msvc.h
x86_64-apple-darwin.h
x86_64-pc-windows-msvc.h
x86_64-pc-linux-gnu.h
Workflow
The workflow changes can be split into two parts, namely building CPython with the JIT enabled and working on the JIT’s implementation.
Building CPython with the JIT
Precompiled JIT stencil files will be stored in the Tools/jit/stencils
directory, with each file name corresponding to its target triple as outlined
above. At build time, we determine whether to use the checked in stencils or to
generate a new stencil for the user’s platform. Specifically, for contributors
with LLVM installed, the build.py
script in Tools/jit/stencils
will allow
them to regenerate the stencil for their platform. Those without LLVM can rely
on the precompiled stencil files directly from the repository.
Working on the JIT’s implementation (or touching JIT files)
In continuous integration (CI), stencil files will be automatically validated and updated when changes
are made to JIT-related files. When a pull request is opened that touches these
files, the jit.yml
workflow, which builds and tests our builds, will run as
usual.
However, as part of this, we will introduce a new step that diffs the current stencils in the repo against those generated in CI. If there is a diff for a platform’s stencil file, a patch file for the updated stencil is generated and the step will fail. Each patch is uploaded to GitHub Actions. After CI is finished running across all platforms, the patches are aggregated into a single patch file for convenience. You can download this aggregated patch, apply it locally, and commit the updated stencils back to your branch. Then, the subsequent CI run will pass.
Reference Implementation
Key parts of the reference implementation include:
.github/workflows/jit.yml
: The CI workflow responsible for generating stencil patches.Tools/jit/stencils
: The directory where stencils are stored.Tools/jit/_targets
: The code to compile and parse the templates at build time.
Ignoring the stencils themselves and any necessary JIT README changes, the changes to the source code to support reproducible stencil generation and hosting are minimal (around 150 lines of changes).
Rejected Ideas
Several alternative approaches were considered as part of the research and exploration for this PEP. However, the ideas below either involve infrastructural cost, maintenance burden, or a worse overall developer experience.
Using Git submodules
Git submodules are a poor developer experience for hosting stencils because they create a different kind of undesirable friction. For instance, any updates to the JIT would necessitate regenerating the stencils and committing them to a separate repository. This introduces a convoluted process: you must update the stencils in the submodule repository, commit those changes, and then update the submodule reference in the main CPython repository. This disconnect adds unnecessary complexity and overhead, making the process brittle and error-prone for contributors and maintainers.
Using Git subtrees
When using subtrees, the embedded repository becomes part of the main repository, similar to what’s being proposed in this PEP. However, subtrees require additional tooling and steps for maintenance, which adds unnecessary complexity to workflows.
Hosting in a separate repository
While splitting JIT stencils into a separate repository avoids the storage overhead associated with hosting the stencils, it adds complexity to the build process. Additional tooling would be required to fetch the stencils and potentially create additional and unnecessary failure points in the workflow. This separation also makes it harder to ensure consistency between the stencils and the CPython source tree, as updates must be coordinated across the repositories.
Hosting in cloud storage
Hosting stencils in cloud storage like S3 buckets or GitHub raw storage introduces external dependencies, complicating offline development workflows. Also, depending on the provider, this type of hosting comes with additional cost, which we’d like to avoid.
Using Git LFS
Git Large File Storage (LFS) adds a tool dependency for contributors, complicating the development workflow, especially for those who may not already use Git LFS. Git LFS does not work well with offline workflows since files managed by LFS require an internet connection to fetch when checking out specific commits, which is disruptive for even basic Git workflows. Git LFS has some free quota but there are additional costs for exceeding that quota which are also undesirable.
Maintain the status quo with LLVM as a build-time dependency
Retaining LLVM as a build-time dependency upholds the existing barriers to adoption and contribution. Ultimately, this option fails to address the core challenges of accessibility and simplicity, and fails to eliminate the dependency which was deemed undesirable at the Python Core Developer Sprint in the fall (the impetus for this PEP), making it a poor long-term solution.
Copyright
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
Source: https://github.com/python/peps/blob/main/peps/pep-0774.rst
Last modified: 2025-02-01 07:28:42 GMT