Abstract

As PEP 654 ExceptionGroup has come into widespread use across the Python community, some common but awkward patterns have emerged. We therefore propose adding two new methods to exception objects:

• BaseExceptionGroup.flat_exceptions(), returning the ‘leaf’ exceptions as a list, with each traceback composited from any intermediate groups.
• BaseException.preserve_context(), a context manager which saves and restores the self.__context__ attribute of self, so that re-raising the exception within another handler does not overwrite the existing context.

We expect this to enable more concise expression of error handling logic in many medium-complexity cases. Without them, exception-group handlers will continue to discard intermediate tracebacks and mis-handle __context__ exceptions, to the detriment of anyone debugging async code.

Motivation

As exception groups come into widespread use, library authors and end users often write code to process or respond to individual leaf exceptions, for example when implementing middleware, error logging, or response handlers in a web framework.

Searching GitHub found four implementations of flat_exceptions() by various names in the first sixty hits, of which none handle tracebacks.[1] The same search found thirteen cases where .flat_exceptions() could be used. We therefore believe that providing a method on the BaseException type with proper traceback preservation will improve error-handling and debugging experiences across the ecosystem.

The rise of exception groups has also made re-raising exceptions caught by an earlier handler much more common: for example, web-server middleware might unwrap HTTPException if that is the sole leaf of a group:

except* HTTPException as group:
    first, *rest = group.flat_exceptions()  # get the whole traceback :-)
    if not rest:
        raise first
    raise

However, this innocent-seeming code has a problem: raise first will do first.__context__ = group as a side effect. This discards the original context of the error, which may contain crucial information to understand why the exception was raised. In many production apps it also causes tracebacks to balloon from hundreds of lines, to tens or even hundreds of thousands of lines - a volume which makes understanding errors far more difficult than it should be.

A new BaseException.preserve_context() method would be a discoverable, readable, and easy-to-use solution for these cases.

Specification

A new method flat_exceptions() will be added to BaseExceptionGroup, with the following signature:

def flat_exceptions(self, *, fix_tracebacks=True) -> list[BaseException]:
    """
    Return a flat list of all 'leaf' exceptions in the group.

    If fix_tracebacks is True, each leaf will have the traceback replaced
    with a composite so that frames attached to intermediate groups are
    still visible when debugging. Pass fix_tracebacks=False to disable
    this modification, e.g. if you expect to raise the group unchanged.
    """

A new method preserve_context() will be added to BaseException, with the following signature:

def preserve_context(self) -> contextlib.AbstractContextManager[Self]:
    """
    Context manager that preserves the exception's __context__ attribute.

    When entering the context, the current values of __context__ is saved.
    When exiting, the saved value is restored, which allows raising an
    exception inside an except block without changing its context chain.
    """

Usage example:

# We're an async web framework, where user code can raise an HTTPException
# to return a particular HTTP error code to the client. However, it may
# (or may not) be raised inside a TaskGroup, so we need to use `except*`;
# and if there are *multiple* such exceptions we'll treat that as a bug.
try:
    user_code_here()
except* HTTPException as group:
    first, *rest = group.flat_exceptions()
    if rest:
        raise  # handled by internal-server-error middleware
    ... # logging, cache updates, etc.
    with first.preserve_context():
        raise first

Without .preserve_context(), this code would have to either:

• arrange for the exception to be raised after the except* block, making code difficult to follow in nontrivial cases, or
• discard the existing __context__ of the first exception, replacing it with an ExceptionGroup which is simply an implementation detail, or
• use try/except instead of except*, handling the possibility that the group doesn’t contain an HTTPException at all,[2] or
• implement the semantics of .preserve_context() inline; while this is not literally unheard-of, it remains very rare.

Backwards Compatibility

Adding new methods to built-in classes, especially those as widely used as BaseException, can have substantial impacts. However, GitHub search shows no collisions for these method names (zero hits and three unrelated hits respectively). If user-defined methods with these names exist in private code they will shadow those proposed in the PEP, without changing runtime behavior.

How to Teach This

Working with exception groups is an intermediate-to-advanced topic, unlikely to arise for beginning programmers. We therefore suggest teaching this topic via documentation, and via just-in-time feedback from static analysis tools. In intermediate classes, we recommend teaching .flat_exceptions() together with the .split() and .subgroup() methods, and mentioning .preserve_context() as an advanced option to address specific pain points.

Both the API reference and the existing ExceptionGroup tutorial should be updated to demonstrate and explain the new methods. The tutorial should include examples of common patterns where .flat_exceptions() and .preserve_context() help simplify error handling logic. Downstream libraries which often use exception groups could include similar docs.

We have also designed lint rules for inclusion in flake8-async which will suggest using .flat_exceptions() when iterating over group.exceptions or re-raising a leaf exception, and suggest using .preserve_context() when re-raising a leaf exception inside an except* block would override any existing context.

Reference Implementation

While the methods on built-in exceptions will be implemented in C if this PEP is accepted, we hope that the following Python implementation will be useful on older versions of Python, and can demonstrate the intended semantics.

We have found these helper functions quite useful when working with ExceptionGroups in a large production codebase.

import copy
import types
from types import TracebackType


def flat_exceptions(
    self: BaseExceptionGroup, *, fix_traceback: bool = True
) -> list[BaseException]:
    """
    Return a flat list of all 'leaf' exceptions.

    If fix_tracebacks is True, each leaf will have the traceback replaced
    with a composite so that frames attached to intermediate groups are
    still visible when debugging. Pass fix_tracebacks=False to disable
    this modification, e.g. if you expect to raise the group unchanged.
    """

    def _flatten(group: BaseExceptionGroup, parent_tb: TracebackType | None = None):
        group_tb = group.__traceback__
        combined_tb = _combine_tracebacks(parent_tb, group_tb)
        result = []
        for exc in group.exceptions:
            if isinstance(exc, BaseExceptionGroup):
                result.extend(_flatten(exc, combined_tb))
            elif fix_tracebacks:
                tb = _combine_tracebacks(combined_tb, exc.__traceback__)
                result.append(exc.with_traceback(tb))
            else:
                result.append(exc)
        return result

    return _flatten(self)


def _combine_tracebacks(
    tb1: TracebackType | None,
    tb2: TracebackType | None,
) -> TracebackType | None:
    """
    Combine two tracebacks, putting tb1 frames before tb2 frames.

    If either is None, return the other.
    """
    if tb1 is None:
        return tb2
    if tb2 is None:
        return tb1

    # Convert tb1 to a list of frames
    frames = []
    current = tb1
    while current is not None:
        frames.append((current.tb_frame, current.tb_lasti, current.tb_lineno))
        current = current.tb_next

    # Create a new traceback starting with tb2
    new_tb = tb2

    # Add frames from tb1 to the beginning (in reverse order)
    for frame, lasti, lineno in reversed(frames):
        new_tb = types.TracebackType(
            tb_next=new_tb, tb_frame=frame, tb_lasti=lasti, tb_lineno=lineno
        )

    return new_tb

class preserve_context:
    def __init__(self, exc: BaseException):
        self.__exc = exc
        self.__context = exc.__context__

    def __enter__(self):
        return self.__exc

    def __exit__(self, exc_type, exc_value, traceback):
        assert exc_value is self.__exc, f"did not raise the expected exception {self.__exc!r}"
        exc_value.__context__ = self.__context
        del self.__context  # break gc cycle

Rejected Ideas

def as_group(self):
    if not isinstance(self, BaseExceptionGroup):
        return BaseExceptionGroup("", [self])
    return self

Footnotes

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.