Py_IsFinalizing is Insufficient

The docs currently recommend Py_IsFinalizing() to guard against termination of the thread:

Calling this function from a thread when the runtime is finalizing will terminate the thread, even if the thread was not created by Python. You can use Py_IsFinalizing() or sys.is_finalizing() to check if the interpreter is in process of being finalized before calling this function to avoid unwanted termination.

Unfortunately, this isn’t correct, because of time-of-call to time-of-use issues; the interpreter might not be finalizing during the call to Py_IsFinalizing(), but it might start finalizing immediately afterwards, which would cause the attachment of a thread state to hang the thread.

Daemon Threads Can Break Finalization

When acquiring locks, it’s extremely important to detach the thread state to prevent deadlocks. This is true on both the with-GIL and free-threaded builds.

When the GIL is enabled, a deadlock can occur pretty easily when acquiring a lock if the GIL wasn’t released; thread A grabs a lock, and starts waiting on its thread state to attach, while thread B holds the GIL and is waiting on the lock. A similar deadlock can occur on the free-threaded build during stop-the-world pauses when running the garbage collector.

This affects CPython itself, and there’s not much that can be done to fix it with the current API. For example, python/cpython#129536 remarks that the ssl module can emit a fatal error when used at finalization, because a daemon thread got hung while holding the lock for sys.stderr, and then a finalizer tried to write to it. Ideally, a thread should be able to temporarily prevent the interpreter from hanging it while it holds the lock.

However, it’s generally unsafe to acquire Python locks (for example, threading.Lock) in finalizers, because the garbage collector might run while the lock is held, which would deadlock if another finalizer tried to acquire the lock. This does not apply to many C locks, such as with sys.stderr, because Python code cannot be run while the lock is held. This PEP intends to fix this problem for C locks, not Python locks.

Daemon Threads are not the Problem

Prior to this PEP, deprecating daemon threads was discussed extensively. Daemon threads technically cause many of the issues outlined in this proposal, so removing daemon threads could be seen as a potential solution. The main argument for removing daemon threads is that they’re a large cause of problems in the interpreter [1].

Except that daemon threads don’t actually work reliably. They’re attempting to run and use Python interpreter resources after the runtime has been shut down upon runtime finalization. As in they have pointers to global state for the interpreter.

However, in practice, daemon threads are useful for simplifying many threading applications in Python, and since the program is about to close in most cases, it’s not worth the added complexity to try and gracefully shut down a thread [2].

When I’ve needed daemon threads, it’s usually been the case of “Long-running, uninterruptible, third-party task” in terms of the examples in the linked issue. Basically I’ve had something that I need running in the background, but I have no easy way to terminate it short of process termination. Unfortunately, I’m on Windows, so signal.pthread_kill isn’t an option. I guess I could use the Windows Terminate Thread API, but it’s a lot of work to wrap it myself compared to just letting process termination handle things.

Finally, removing Python-level daemon threads does not fix the whole problem. As noted by this PEP, extension modules are free to create their own threads and attach thread states for them. Similar to daemon threads, Python doesn’t try and join them during finalization, so trying to remove daemon threads as a whole would involve trying to remove them from the C API, which would require a much more massive API change than what is currently being proposed [3].

Realize however that even if we get rid of daemon threads, extension module code can and does spawn its own threads that are not tracked by Python. … Those are realistically an alternate form of daemon thread … and those are never going to be forbidden.

Joining the Thread isn’t Always a Good Idea

Even in daemon threads, it’s generally possible to prevent hanging of non-Python threads through atexit functions. A thread could be started by some C function, and then as long as that thread is joined by atexit, then the thread won’t hang.

atexit isn’t always an option for a function, because to call it, it needs to already have an attached thread state for the thread. If there’s no guarantee of that, then atexit.register() cannot be safely called without the risk of hanging the thread. This shifts the contract of joining the thread to the caller rather than the callee, which again, isn’t reliable enough in practice to be a viable solution.

For example, large C++ applications might want to expose an interface that can call Python code. To do this, a C++ API would take a Python object, and then call PyGILState_Ensure() to safely interact with it (for example, by calling it). If the interpreter is finalizing or has shut down, then the thread is hung, disrupting the C++ stream of calls.

Finalization Behavior for PyGILState_Ensure Cannot Change

There will always have to be a point in a Python program where PyGILState_Ensure() can no longer attach a thread state. If the interpreter is long dead, then Python obviously can’t give a thread a way to invoke it. PyGILState_Ensure() doesn’t have any meaningful way to return a failure, so it has no choice but to terminate the thread or emit a fatal error, as noted in python/cpython#124622:

I think a new GIL acquisition and release C API would be needed. The way the existing ones get used in existing C code is not amenible to suddenly bolting an error state onto; none of the existing C code is written that way. After the call they always just assume they have the GIL and can proceed. The API was designed as “it’ll block and only return once it has the GIL” without any other option.

For this reason, we can’t make any real changes to how PyGILState_Ensure() works during finalization, because it would break existing code.

PyGILState_Ensure Generally Crashes During Finalization

At the time of writing, the current behavior of PyGILState_Ensure() does not always match the documentation. Instead of hanging the thread during finalization as previously noted, it’s possible for it to crash with a segmentation fault. This is a known issue that could be fixed in CPython, but it’s definitely worth noting here, because acceptance and implementation of this PEP will likely fix the existing crashes caused by PyGILState_Ensure().

The Term “GIL” is Tricky for Free-threading

A large issue with the term “GIL” in the C API is that it is semantically misleading. This was noted in python/cpython#127989, created by the authors of this PEP:

The biggest issue is that for free-threading, there is no GIL, so users erroneously call the C API inside Py_BEGIN_ALLOW_THREADS blocks or omit PyGILState_Ensure in fresh threads.

Again, PyGILState_Ensure() gets an attached thread state for the thread on both with-GIL and free-threaded builds. An attached thread state is always needed to call the C API, so PyGILState_Ensure() still needs to be called on free-threaded builds, but with a name like “ensure GIL”, it’s not immediately clear that that’s true.

Weak References

This proposal also comes with weak references to an interpreter that don’t prevent it from shutting down, but can be promoted to a strong reference when the user decides that they want to call the C API. If an interpreter is destroyed or past the point where it can create strong references, promotion of a weak reference will fail.

A weak reference will typically live much longer than a strong reference. This is useful for many of the asynchronous situations stated previously, where the thread itself shouldn’t prevent the desired interpreter from shutting down, but also allow the thread to execute Python when needed.

For example, a (non-reentrant) event handler may store a weak interpreter reference in its void *arg parameter, and then that weak reference will be promoted to a strong reference when it’s time to call Python code.

Strong Interpreter References

type PyInterpreterRef

An opaque, strong reference to an interpreter. The interpreter will wait until a strong reference has been released before shutting down.

This type is guaranteed to be pointer-sized.

int PyInterpreterRef_Get(PyInterpreterRef *ref)

Acquire a strong reference to the current interpreter.

On success, this function returns 0 and sets ref to a strong reference to the interpreter, and returns -1 with an exception set on failure.

Failure typically indicates that the interpreter has already finished waiting on strong references.

The caller must hold an attached thread state.

int PyInterpreterRef_Main(PyInterpreterRef *ref)

Acquire a strong reference to the main interpreter.

This function only exists for special cases where a specific interpreter can’t be saved. Prefer safely acquiring a reference through PyInterpreterRef_Get() whenever possible.

On success, this function will return 0 and set ref to a strong reference, and on failure, this function will return -1.

Failure typically indicates that the main interpreter has already finished waiting on its reference count.

The caller does not need to hold an attached thread state.

PyInterpreterState *PyInterpreterRef_AsInterpreter(PyInterpreterRef ref)

Return the interpreter denoted by ref.

This function cannot fail, and the caller doesn’t need to hold an attached thread state.

PyInterpreterRef PyInterpreterRef_Dup(PyInterpreterRef ref)

Duplicate a strong reference to an interpreter.

This function cannot fail, and the caller doesn’t need to hold an attached thread state.

void PyInterpreterRef_Close(PyInterpreterRef ref)

Release a strong reference to an interpreter, allowing it to shut down if there are no references left.

This function cannot fail, and the caller doesn’t need to hold an attached thread state.

Weak Interpreter References

type PyInterpreterWeakRef

An opaque, weak reference to an interpreter. The interpreter will not wait for the reference to be released before shutting down.

This type is guaranteed to be pointer-sized.

int PyInterpreterWeakRef_Get(PyInterpreterWeakRef *wref)

Acquire a weak reference to the current interpreter.

This function is generally meant to be used in tandem with PyInterpreterWeakRef_AsStrong().

On success, this function returns 0 and sets wref to a weak reference to the interpreter, and returns -1 with an exception set on failure.

The caller must hold an attached thread state.

PyInterpreterWeakRef PyInterpreterWeakRef_Dup(PyInterpreterWeakRef wref)

Duplicate a weak reference to an interpreter.

This function cannot fail, and the caller doesn’t need to hold an attached thread state.

int PyInterpreterWeakRef_AsStrong(PyInterpreterWeakRef wref, PyInterpreterRef *ref)

Acquire a strong reference to an interpreter through a weak reference.

On success, this function returns 0 and sets ref to a strong reference to the interpreter denoted by wref.

If the interpreter no longer exists or has already finished waiting for its reference count to reach zero, then this function returns -1 without an exception set.

This function is not safe to call in a re-entrant signal handler.

The caller does not need to hold an attached thread state.

void PyInterpreterWeakRef_Close(PyInterpreterWeakRef wref)

Release a weak reference to an interpreter.

This function cannot fail, and the caller doesn’t need to hold an attached thread state.

Example: A Library Interface

Imagine that you’re developing a C library for logging. You might want to provide an API that allows users to log to a Python file object.

With this PEP, you’d implement it like this:

int
LogToPyFile(PyInterpreterWeakRef wref,
            PyObject *file,
            const char *text)
{
    PyInterpreterRef ref;
    if (PyInterpreterWeakRef_AsStrong(wref, &ref) < 0) {
        /* Python interpreter has shut down */
        return -1;
    }

    PyThreadRef thread_ref;
    if (PyThreadState_Ensure(ref, &thread_ref) < 0) {
        PyInterpreterRef_Close(ref);
        puts("Out of memory.\n", stderr);
        return -1;
    }

    char *to_write = do_some_text_mutation(text);
    int res = PyFile_WriteString(to_write, file);
    free(to_write);
    PyErr_Print();

    PyThreadState_Release(thread_ref);
    PyInterpreterRef_Close(ref);
    return res < 0;
}

If you were to use PyGILState_Ensure() for this case, then your thread would hang if the interpreter were to be finalizing at that time!

Additionally, the API supports subinterpreters. If you were to assume that the main interpreter created the file object (via PyGILState_Ensure()), then using file objects owned by a subinterpreter could possibly crash.

Example: A Single-threaded Ensure

This example shows acquiring a lock in a Python method.

If this were to be called from a daemon thread, then the interpreter could hang the thread while reattaching the thread state, leaving us with the lock held. Any future finalizer that wanted to acquire the lock would be deadlocked!

static PyObject *
my_critical_operation(PyObject *self, PyObject *unused)
{
    assert(PyThreadState_GetUnchecked() != NULL);
    PyInterpreterRef ref;
    if (PyInterpreterRef_Get(&ref) < 0) {
        /* Python interpreter has shut down */
        return NULL;
    }

    Py_BEGIN_ALLOW_THREADS;
    acquire_some_lock();

    /* Do something while holding the lock.
       The interpreter won't finalize during this period. */
    // ...

    release_some_lock();
    Py_END_ALLOW_THREADS;
    PyInterpreterRef_Close(ref);
    Py_RETURN_NONE;
}

Example: Transitioning From the Legacy Functions

The following code uses the PyGILState APIs:

static int
thread_func(void *arg)
{
    PyGILState_STATE gstate = PyGILState_Ensure();
    /* It's not an issue in this example, but we just attached
       a thread state for the main interpreter. If my_method() was
       originally called in a subinterpreter, then we would be unable
       to safely interact with any objects from it. */
    if (PyRun_SimpleString("print(42)") < 0) {
        PyErr_Print();
    }
    PyGILState_Release(gstate);
    return 0;
}

static PyObject *
my_method(PyObject *self, PyObject *unused)
{
    PyThread_handle_t handle;
    PyThead_indent_t indent;

    if (PyThread_start_joinable_thread(thread_func, NULL, &ident, &handle) < 0) {
        return NULL;
    }
    Py_BEGIN_ALLOW_THREADS;
    PyThread_join_thread(handle);
    Py_END_ALLOW_THREADS;
    Py_RETURN_NONE;
}

This is the same code, rewritten to use the new functions:

static int
thread_func(void *arg)
{
    PyInterpreterRef interp = (PyInterpreterRef)arg;
    PyThreadRef thread_ref;
    if (PyThreadState_Ensure(interp, &thread_ref) < 0) {
        PyInterpreterRef_Close(interp);
        return -1;
    }
    if (PyRun_SimpleString("print(42)") < 0) {
        PyErr_Print();
    }
    PyThreadState_Release(thread_ref);
    PyInterpreterRef_Close(interp);
    return 0;
}

static PyObject *
my_method(PyObject *self, PyObject *unused)
{
    PyThread_handle_t handle;
    PyThead_indent_t indent;

    PyInterpreterRef ref;
    if (PyInterpreterRef_Get(&ref) < 0) {
        return NULL;
    }

    if (PyThread_start_joinable_thread(thread_func, (void *)ref, &ident, &handle) < 0) {
        PyInterpreterRef_Close(ref);
        return NULL;
    }
    Py_BEGIN_ALLOW_THREADS
    PyThread_join_thread(handle);
    Py_END_ALLOW_THREADS
    Py_RETURN_NONE;
}

Example: A Daemon Thread

With this PEP, daemon threads are very similar to how non-Python threads work in the C API today. After calling PyThreadState_Ensure(), simply release the interpreter reference, allowing the interpreter to shut down.

static int
thread_func(void *arg)
{
    PyInterpreterRef ref = (PyInterpreterRef)arg;
    PyThreadRef thread_ref;
    if (PyThreadState_Ensure(ref, &thread_ref) < 0) {
        PyInterpreterRef_Close(ref);
        return -1;
    }
    /* Release the interpreter reference, allowing it to
       finalize. This means that print(42) can hang this thread. */
    PyInterpreterRef_Close(ref);
    if (PyRun_SimpleString("print(42)") < 0) {
        PyErr_Print();
    }
    PyThreadState_Release(thread_ref);
    return 0;
}

static PyObject *
my_method(PyObject *self, PyObject *unused)
{
    PyThread_handle_t handle;
    PyThead_indent_t indent;

    PyInterpreterRef ref;
    if (PyInterpreterRef_Get(&ref) < 0) {
        return NULL;
    }

    if (PyThread_start_joinable_thread(thread_func, (void *)ref, &ident, &handle) < 0) {
        PyInterpreterRef_Close(ref);
        return NULL;
    }
    Py_RETURN_NONE;
}

Example: An Asynchronous Callback

In some cases, the thread might not ever start, such as in a callback. We can’t use a strong reference here, because a strong reference would deadlock the interpreter if it’s not released.

typedef struct {
    PyInterpreterWeakRef wref;
} ThreadData;

static int
async_callback(void *arg)
{
    ThreadData *data = (ThreadData *)arg;
    PyInterpreterWeakRef wref = data->wref;
    PyInterpreterRef ref;
    if (PyInterpreterWeakRef_AsStrong(wref, &ref) < 0) {
        fputs("Python has shut down!\n", stderr);
        return -1;
    }

    PyThreadRef thread_ref;
    if (PyThreadState_Ensure(ref, &thread_ref) < 0) {
        PyInterpreterRef_Close(ref);
        return -1;
    }
    if (PyRun_SimpleString("print(42)") < 0) {
        PyErr_Print();
    }
    PyThreadState_Release(thread_ref);
    PyInterpreterRef_Close(ref);
    return 0;
}

static PyObject *
setup_callback(PyObject *self, PyObject *unused)
{
    // Weak reference to the interpreter. It won't wait on the callback
    // to finalize.
    ThreadData *tdata = PyMem_RawMalloc(sizeof(ThreadData));
    if (tdata == NULL) {
        PyErr_NoMemory();
        return NULL;
    }
    PyInterpreterWeakRef wref;
    if (PyInterpreterWeakRef_Get(&wref) < 0) {
        PyMem_RawFree(tdata);
        return NULL;
    }
    tdata->wref = wref;
    register_callback(async_callback, tdata);

    Py_RETURN_NONE;
}

Example: Calling Python Without a Callback Parameter

There are a few cases where callback functions don’t take a callback parameter (void *arg), so it’s impossible to acquire a reference to any specific interpreter. The solution to this problem is to acquire a reference to the main interpreter through PyInterpreterRef_Main().

But wait, won’t that break with subinterpreters, per PyGILState_Ensure Doesn’t Guess the Correct Interpreter? Fortunately, since the callback has no callback parameter, it’s not possible for the caller to pass any objects or interpreter-specific data, so it’s completely safe to choose the main interpreter here.

static void
call_python(void)
{
    PyInterpreterRef ref;
    if (PyInterpreterRef_Main(&ref) < 0) {
        fputs("Python has shut down!", stderr);
        return;
    }

    PyThreadRef thread_ref;
    if (PyThreadState_Ensure(ref, &thread_ref) < 0) {
        PyInterpreterRef_Close(ref);
        return -1;
    }
    if (PyRun_SimpleString("print(42)") < 0) {
        PyErr_Print();
    }
    PyThreadState_Release(thread_ref);
    PyInterpreterRef_Close(ref);
    return 0;
}

Interpreter-State Pointers for Reference Counting

Originally, this PEP specified PyInterpreterState_Hold() and PyInterpreterState_Release() for managing strong references to an interpreter, alongside PyInterpreterState_Lookup() which converted interpreter IDs (weak references) to strong references.

In the end, this was rejected, primarily because it was needlessly confusing. Interpreter states hadn’t ever had a reference count prior, so there was a lack of intuition about when and where something was a strong reference. The PyInterpreterRef and PyInterpreterWeakRef types seem a lot clearer.

Interpreter IDs for Reference Counting

Some iterations of this API took an int64_t interp_id parameter instead of PyInterpreterState *interp, because interpreter IDs cannot be concurrently deleted and cause use-after-free violations. The reference counting APIs in this PEP sidestep this issue anyway, but an interpreter ID have the advantage of requiring less magic:

• Nearly all existing interpreter APIs already return a PyInterpreterState pointer, not an interpreter ID. Functions like PyThreadState_GetInterpreter() would have to be accompanied by frustrating calls to PyInterpreterState_GetID().
• Threads typically take a void *arg parameter, not an int64_t arg. As such, passing a reference requires much less boilerplate for the user, because an additional structure definition or heap allocation would be needed to store the interpreter ID. This is especially an issue on 32-bit systems, where void * is too small for an int64_t.
• To retain usability, interpreter ID APIs would still need to keep a reference count, otherwise the interpreter could be finalizing before the non-Python thread gets a chance to attach. The problem with using an interpreter ID is that the reference count has to be “invisible”; it must be tracked elsewhere in the interpreter, likely being more complex than PyInterpreterRef_Get(). There’s also a lack of intuition that a standalone integer could have such a thing as a reference count.