Following system colour scheme Selected dark colour scheme Selected light colour scheme

Python Enhancement Proposals

PEP 774 – Removing the LLVM requirement for JIT builds

Author:
Savannah Ostrowski <savannah at python.org>
Discussions-To:
Discourse thread
Status:
Draft
Type:
Standards Track
Created:
27-Jan-2025
Python-Version:
3.14

Table of Contents

Abstract

Since Python 3.13, CPython has been able to be configured and built with an experimental just-in-time (JIT) compiler via the --enable-experimental-jit flag on Linux and Mac and --experimental-jit on Windows. To build CPython with the JIT enabled, users are required to have LLVM installed on their machine (initially, with LLVM 16 but more recently, with LLVM 19). LLVM is responsible for generating stencils that are essential to our copy-and-patch JIT (see PEP 744). These stencils are predefined, architecture-specific templates that are used to generate machine code at runtime.

This PEP proposes removing the LLVM build-time dependency for JIT-enabled builds by hosting the generated stencils in the CPython repository. This approach allows us to leverage the checked-in stencils for supported platforms at build time, simplifying the contributor experience and address concerns raised at the Python Core Developer Sprint in September 2024. That said, there is a clear tradeoff to consider, as improved developer experience does come at the cost of increased repository size.

It is important to note that this PEP is not a proposal to accept or reject the JIT itself but rather to determine whether the build-time dependency on LLVM is acceptable for JIT builds moving forward. If this PEP is rejected, we will proceed with the status quo, retaining the LLVM build-time requirement. While this dependency has served the JIT development process effectively thus far, it introduces setup complexity and additional challenges that this PEP seeks to alleviate.

Motivation

At the Python Core Developer Sprint that took place in September 2024, there was discussion about the next steps for the JIT - a related discussion also took place on GitHub. As part of that discussion, there was also a clear appetite for removing the LLVM requirement for JIT builds in preparation for shipping the JIT off by default in 3.14. The consensus at the sprint was that it would be sufficient to provide pre-generated stencils for non-debug builds for Tier 1 platforms and that checking these files into the CPython repo would be adequate for the limited number of platforms (though more options have been explored; see Rejected Ideas).

Currently, building CPython with the JIT requires LLVM as a build-time dependency. Despite not being exposed to end users, this dependency is suboptimal. Requiring LLVM adds a setup burden for developers and those who wish to build CPython with the JIT enabled. Depending on the operating system, the version of LLVM shipped with the OS may differ from that required by our JIT builds, which introduces additional complexity to troubleshoot and resolve. With few core developers currently contributing to and maintaining the JIT, we also want to make sure that the friction to work on JIT-related code is minimized as much as possible.

With the proposed approach, hosting pre-compiled stencils for supported architectures can be generated in advance, stored in a central location, and automatically used during builds. This approach ensures reproducible builds, making the JIT a more stable and sustainable part of CPython’s future.

Rationale

This PEP proposes checking JIT stencils directly into the CPython repo as the best path forward for eliminating our build-time dependency on LLVM.

This approach:

  • Provides the best end-to-end experience for those looking to build CPython with the JIT
  • Lessens the barrier to entry for those looking to contribute to the JIT
  • Ensures builds remain reproducible and consistent across platforms without relying on external infrastructure or download mechanisms
  • Eliminates variability introduced by network conditions or potential discrepancies between hosted files and the CPython repository state, and
  • Subjects stencils to the same review processes we have for all other JIT-related code

However, this approach does result in a slight increase in overall repository size. Comparing repo growth on commits over the past 90 days, the difference between the actual commits and the same commits with stencils added amounts to a difference of 0.03 MB per stencil file. This is a small increase in the context of the overall repository size, which has grown by 2.55 MB in the same time period. For six stencil files, this amounts to an upper bound of 0.18 MB. The current total size of the stencil files for all six platforms is 7.2 MB.

These stencils could become larger in the future with changes to register allocation, which would introduce 5-6 variants per instruction in each stencil file (5-6x larger). However, if we ended up going this route, there are additional modifications we could make to stencil files that could help counteract this size increase (e.g., stripping comments, minimizing the stencils).

Specification

This specification outlines the proposed changes to remove the build-time dependency on LLVM and the contributor experience if this PEP is accepted.

Repository changes

The CPython repository would now host the pre-compiled JIT stencils in a new subdirectory in Tools/jit called stencils/. At present, the JIT is tested and built for six platforms, so to start, we’d check in six stencil files. In the future, we may check in additional stencil files if support for additional platforms is desired or relevant.

cpython/
    Tools/
        jit/
            stencils/
                aarch64-apple-darwin.h
                aarch64-unknown-linux-gnu.h
                i686-pc-windows-msvc.h
                x86_64-apple-darwin.h
                x86_64-pc-windows-msvc.h
                x86_64-pc-linux-gnu.h

Workflow

The workflow changes can be split into two parts, namely building CPython with the JIT enabled and working on the JIT’s implementation.

Building CPython with the JIT

Precompiled JIT stencil files will be stored in the Tools/jit/stencils directory, with each file name corresponding to its target triple as outlined above. At build time, we determine whether to use the checked in stencils or to generate a new stencil for the user’s platform. Specifically, for contributors with LLVM installed, the build.py script in Tools/jit/stencils will allow them to regenerate the stencil for their platform. Those without LLVM can rely on the precompiled stencil files directly from the repository.

Working on the JIT’s implementation (or touching JIT files)

In continuous integration (CI), stencil files will be automatically validated and updated when changes are made to JIT-related files. When a pull request is opened that touches these files, the jit.yml workflow, which builds and tests our builds, will run as usual.

However, as part of this, we will introduce a new step that diffs the current stencils in the repo against those generated in CI. If there is a diff for a platform’s stencil file, a patch file for the updated stencil is generated and the step will fail. Each patch is uploaded to GitHub Actions. After CI is finished running across all platforms, the patches are aggregated into a single patch file for convenience. You can download this aggregated patch, apply it locally, and commit the updated stencils back to your branch. Then, the subsequent CI run will pass.

Reference Implementation

Key parts of the reference implementation include:

Ignoring the stencils themselves and any necessary JIT README changes, the changes to the source code to support reproducible stencil generation and hosting are minimal (around 150 lines of changes).

Rejected Ideas

Several alternative approaches were considered as part of the research and exploration for this PEP. However, the ideas below either involve infrastructural cost, maintenance burden, or a worse overall developer experience.

Using Git submodules

Git submodules are a poor developer experience for hosting stencils because they create a different kind of undesirable friction. For instance, any updates to the JIT would necessitate regenerating the stencils and committing them to a separate repository. This introduces a convoluted process: you must update the stencils in the submodule repository, commit those changes, and then update the submodule reference in the main CPython repository. This disconnect adds unnecessary complexity and overhead, making the process brittle and error-prone for contributors and maintainers.

Using Git subtrees

When using subtrees, the embedded repository becomes part of the main repository, similar to what’s being proposed in this PEP. However, subtrees require additional tooling and steps for maintenance, which adds unnecessary complexity to workflows.

Hosting in a separate repository

While splitting JIT stencils into a separate repository avoids the storage overhead associated with hosting the stencils, it adds complexity to the build process. Additional tooling would be required to fetch the stencils and potentially create additional and unnecessary failure points in the workflow. This separation also makes it harder to ensure consistency between the stencils and the CPython source tree, as updates must be coordinated across the repositories.

Hosting in cloud storage

Hosting stencils in cloud storage like S3 buckets or GitHub raw storage introduces external dependencies, complicating offline development workflows. Also, depending on the provider, this type of hosting comes with additional cost, which we’d like to avoid.

Using Git LFS

Git Large File Storage (LFS) adds a tool dependency for contributors, complicating the development workflow, especially for those who may not already use Git LFS. Git LFS does not work well with offline workflows since files managed by LFS require an internet connection to fetch when checking out specific commits, which is disruptive for even basic Git workflows. Git LFS has some free quota but there are additional costs for exceeding that quota which are also undesirable.

Maintain the status quo with LLVM as a build-time dependency

Retaining LLVM as a build-time dependency upholds the existing barriers to adoption and contribution. Ultimately, this option fails to address the core challenges of accessibility and simplicity, and fails to eliminate the dependency which was deemed undesirable at the Python Core Developer Sprint in the fall (the impetus for this PEP), making it a poor long-term solution.


Source: https://github.com/python/peps/blob/main/peps/pep-0774.rst

Last modified: 2025-02-01 07:28:42 GMT