PEP 344 – Exception Chaining and Embedded Tracebacks
- Author:
- Ka-Ping Yee
- Status:
- Superseded
- Type:
- Standards Track
- Created:
- 12-May-2005
- Python-Version:
- 2.5
- Post-History:
Table of Contents
- Numbering Note
- Abstract
- Motivation
- History
- Rationale
- Implicit Exception Chaining
- Explicit Exception Chaining
- Traceback Attribute
- Enhanced Reporting
- C API
- Compatibility
- Open Issue: Extra Information
- Open Issue: Suppressing Context
- Open Issue: Limiting Exception Types
- Open Issue: yield
- Open Issue: Garbage Collection
- Possible Future Compatible Changes
- Possible Future Incompatible Changes
- Acknowledgements
- References
- Copyright
Numbering Note
This PEP has been renumbered to PEP 3134. The text below is the last version submitted under the old number.
Abstract
This PEP proposes three standard attributes on exception instances: the
__context__
attribute for implicitly chained exceptions, the
__cause__
attribute for explicitly chained exceptions, and the
__traceback__
attribute for the traceback. A new raise ... from
statement sets the __cause__
attribute.
Motivation
During the handling of one exception (exception A), it is possible that another
exception (exception B) may occur. In today’s Python (version 2.4), if this
happens, exception B is propagated outward and exception A is lost. In order
to debug the problem, it is useful to know about both exceptions. The
__context__
attribute retains this information automatically.
Sometimes it can be useful for an exception handler to intentionally re-raise
an exception, either to provide extra information or to translate an exception
to another type. The __cause__
attribute provides an explicit way to
record the direct cause of an exception.
In today’s Python implementation, exceptions are composed of three parts: the
type, the value, and the traceback. The sys
module, exposes the current
exception in three parallel variables, exc_type
, exc_value
, and
exc_traceback
, the sys.exc_info()
function returns a tuple of these
three parts, and the raise
statement has a three-argument form accepting
these three parts. Manipulating exceptions often requires passing these three
things in parallel, which can be tedious and error-prone. Additionally, the
except
statement can only provide access to the value, not the traceback.
Adding the __traceback__
attribute to exception values makes all the
exception information accessible from a single place.
History
Raymond Hettinger [1] raised the issue of masked exceptions on Python-Dev in
January 2003 and proposed a PyErr_FormatAppend()
function that C modules
could use to augment the currently active exception with more information.
Brett Cannon [2] brought up chained exceptions again in June 2003, prompting
a long discussion.
Greg Ewing [3] identified the case of an exception occurring in a finally
block during unwinding triggered by an original exception, as distinct from
the case of an exception occurring in an except
block that is handling the
original exception.
Greg Ewing [4] and Guido van Rossum [5], and probably others, have
previously mentioned adding a traceback attribute to Exception
instances.
This is noted in PEP 3000.
This PEP was motivated by yet another recent Python-Dev reposting of the same ideas [6] [7].
Rationale
The Python-Dev discussions revealed interest in exception chaining for two quite different purposes. To handle the unexpected raising of a secondary exception, the exception must be retained implicitly. To support intentional translation of an exception, there must be a way to chain exceptions explicitly. This PEP addresses both.
Several attribute names for chained exceptions have been suggested on
Python-Dev [2], including cause
, antecedent
, reason
, original
,
chain
, chainedexc
, xc_chain
, excprev
, previous
and
precursor
. For an explicitly chained exception, this PEP suggests
__cause__
because of its specific meaning. For an implicitly chained
exception, this PEP proposes the name __context__
because the intended
meaning is more specific than temporal precedence but less specific than
causation: an exception occurs in the context of handling another exception.
This PEP suggests names with leading and trailing double-underscores for these three attributes because they are set by the Python VM. Only in very special cases should they be set by normal assignment.
This PEP handles exceptions that occur during except
blocks and
finally
blocks in the same way. Reading the traceback makes it clear
where the exceptions occurred, so additional mechanisms for distinguishing
the two cases would only add unnecessary complexity.
This PEP proposes that the outermost exception object (the one exposed for
matching by except
clauses) be the most recently raised exception for
compatibility with current behaviour.
This PEP proposes that tracebacks display the outermost exception last, because this would be consistent with the chronological order of tracebacks (from oldest to most recent frame) and because the actual thrown exception is easier to find on the last line.
To keep things simpler, the C API calls for setting an exception will not
automatically set the exception’s __context__
. Guido van Rossum has
expressed concerns with making such changes [8].
As for other languages, Java and Ruby both discard the original exception when
another exception occurs in a catch/rescue
or finally/ensure
clause.
Perl 5 lacks built-in structured exception handling. For Perl 6, RFC number
88 [9] proposes an exception mechanism that implicitly retains chained
exceptions in an array named @@
. In that RFC, the most recently raised
exception is exposed for matching, as in this PEP; also, arbitrary expressions
(possibly involving @@
) can be evaluated for exception matching.
Exceptions in C# contain a read-only InnerException
property that may
point to another exception. Its documentation [10] says that “When an
exception X is thrown as a direct result of a previous exception Y, the
InnerException
property of X should contain a reference to Y.” This
property is not set by the VM automatically; rather, all exception
constructors take an optional innerException
argument to set it
explicitly. The __cause__
attribute fulfills the same purpose as
InnerException
, but this PEP proposes a new form of raise
rather than
extending the constructors of all exceptions. C# also provides a
GetBaseException
method that jumps directly to the end of the
InnerException
chain; this PEP proposes no analog.
The reason all three of these attributes are presented together in one proposal
is that the __traceback__
attribute provides convenient access to the
traceback on chained exceptions.
Implicit Exception Chaining
Here is an example to illustrate the __context__
attribute:
def compute(a, b):
try:
a/b
except Exception, exc:
log(exc)
def log(exc):
file = open('logfile.txt') # oops, forgot the 'w'
print >>file, exc
file.close()
Calling compute(0, 0)
causes a ZeroDivisionError
. The compute()
function catches this exception and calls log(exc)
, but the log()
function also raises an exception when it tries to write to a file that wasn’t
opened for writing.
In today’s Python, the caller of compute()
gets thrown an IOError
. The
ZeroDivisionError
is lost. With the proposed change, the instance of
IOError
has an additional __context__
attribute that retains the
ZeroDivisionError
.
The following more elaborate example demonstrates the handling of a mixture of
finally
and except
clauses:
def main(filename):
file = open(filename) # oops, forgot the 'w'
try:
try:
compute()
except Exception, exc:
log(file, exc)
finally:
file.clos() # oops, misspelled 'close'
def compute():
1/0
def log(file, exc):
try:
print >>file, exc # oops, file is not writable
except:
display(exc)
def display(exc):
print ex # oops, misspelled 'exc'
Calling main()
with the name of an existing file will trigger four
exceptions. The ultimate result will be an AttributeError
due to the
misspelling of clos
, whose __context__
points to a NameError
due
to the misspelling of ex
, whose __context__
points to an IOError
due to the file being read-only, whose __context__
points to a
ZeroDivisionError
, whose __context__
attribute is None
.
The proposed semantics are as follows:
- Each thread has an exception context initially set to
None
. - Whenever an exception is raised, if the exception instance does not
already have a
__context__
attribute, the interpreter sets it equal to the thread’s exception context. - Immediately after an exception is raised, the thread’s exception context is set to the exception.
- Whenever the interpreter exits an
except
block by reaching the end or executing areturn
,yield
,continue
, orbreak
statement, the thread’s exception context is set toNone
.
Explicit Exception Chaining
The __cause__
attribute on exception objects is always initialized to
None
. It is set by a new form of the raise
statement:
raise EXCEPTION from CAUSE
which is equivalent to:
exc = EXCEPTION
exc.__cause__ = CAUSE
raise exc
In the following example, a database provides implementations for a few
different kinds of storage, with file storage as one kind. The database
designer wants errors to propagate as DatabaseError
objects so that the
client doesn’t have to be aware of the storage-specific details, but doesn’t
want to lose the underlying error information:
class DatabaseError(StandardError):
pass
class FileDatabase(Database):
def __init__(self, filename):
try:
self.file = open(filename)
except IOError, exc:
raise DatabaseError('failed to open') from exc
If the call to open()
raises an exception, the problem will be reported as
a DatabaseError
, with a __cause__
attribute that reveals the
IOError
as the original cause.
Traceback Attribute
The following example illustrates the __traceback__
attribute:
def do_logged(file, work):
try:
work()
except Exception, exc:
write_exception(file, exc)
raise exc
from traceback import format_tb
def write_exception(file, exc):
...
type = exc.__class__
message = str(exc)
lines = format_tb(exc.__traceback__)
file.write(... type ... message ... lines ...)
...
In today’s Python, the do_logged()
function would have to extract the
traceback from sys.exc_traceback
or sys.exc_info()
[2] and pass both
the value and the traceback to write_exception()
. With the proposed
change, write_exception()
simply gets one argument and obtains the
exception using the __traceback__
attribute.
The proposed semantics are as follows:
- Whenever an exception is caught, if the exception instance does not already
have a
__traceback__
attribute, the interpreter sets it to the newly caught traceback.
Enhanced Reporting
The default exception handler will be modified to report chained exceptions.
The chain of exceptions is traversed by following the __cause__
and
__context__
attributes, with __cause__
taking priority. In keeping
with the chronological order of tracebacks, the most recently raised exception
is displayed last; that is, the display begins with the description of the
innermost exception and backs up the chain to the outermost exception. The
tracebacks are formatted as usual, with one of the lines:
The above exception was the direct cause of the following exception:
or
During handling of the above exception, another exception occurred:
between tracebacks, depending whether they are linked by __cause__
or
__context__
respectively. Here is a sketch of the procedure:
def print_chain(exc):
if exc.__cause__:
print_chain(exc.__cause__)
print '\nThe above exception was the direct cause...'
elif exc.__context__:
print_chain(exc.__context__)
print '\nDuring handling of the above exception, ...'
print_exc(exc)
In the traceback
module, the format_exception
, print_exception
,
print_exc
, and print_last functions
will be updated to accept an
optional chain
argument, True
by default. When this argument is
True
, these functions will format or display the entire chain of
exceptions as just described. When it is False
, these functions will
format or display only the outermost exception.
The cgitb
module should also be updated to display the entire chain of
exceptions.
C API
The PyErr_Set*
calls for setting exceptions will not set the
__context__
attribute on exceptions. PyErr_NormalizeException
will
always set the traceback
attribute to its tb
argument and the
__context__
and __cause__
attributes to None
.
A new API function, PyErr_SetContext(context)
, will help C programmers
provide chained exception information. This function will first normalize the
current exception so it is an instance, then set its __context__
attribute. A similar API function, PyErr_SetCause(cause)
, will set the
__cause__
attribute.
Compatibility
Chained exceptions expose the type of the most recent exception, so they will
still match the same except
clauses as they do now.
The proposed changes should not break any code unless it sets or uses
attributes named __context__
, __cause__
, or __traceback__
on
exception instances. As of 2005-05-12, the Python standard library contains
no mention of such attributes.
Open Issue: Extra Information
Walter Dörwald [11] expressed a desire to attach extra information to an exception during its upward propagation without changing its type. This could be a useful feature, but it is not addressed by this PEP. It could conceivably be addressed by a separate PEP establishing conventions for other informational attributes on exceptions.
Open Issue: Suppressing Context
As written, this PEP makes it impossible to suppress __context__
, since
setting exc.__context__
to None
in an except
or finally
clause
will only result in it being set again when exc
is raised.
Open Issue: Limiting Exception Types
To improve encapsulation, library implementors may want to wrap all implementation-level exceptions with an application-level exception. One could try to wrap exceptions by writing this:
try:
... implementation may raise an exception ...
except:
import sys
raise ApplicationError from sys.exc_value
or this
try:
... implementation may raise an exception ...
except Exception, exc:
raise ApplicationError from exc
but both are somewhat flawed. It would be nice to be able to name the current
exception in a catch-all except
clause, but that isn’t addressed here.
Such a feature would allow something like this:
try:
... implementation may raise an exception ...
except *, exc:
raise ApplicationError from exc
Open Issue: yield
The exception context is lost when a yield
statement is executed; resuming
the frame after the yield
does not restore the context. Addressing this
problem is out of the scope of this PEP; it is not a new problem, as
demonstrated by the following example:
>>> def gen():
... try:
... 1/0
... except:
... yield 3
... raise
...
>>> g = gen()
>>> g.next()
3
>>> g.next()
TypeError: exceptions must be classes, instances, or strings
(deprecated), not NoneType
Open Issue: Garbage Collection
The strongest objection to this proposal has been that it creates cycles between exceptions and stack frames [12]. Collection of cyclic garbage (and therefore resource release) can be greatly delayed:
>>> try:
>>> 1/0
>>> except Exception, err:
>>> pass
will introduce a cycle from err -> traceback -> stack frame -> err, keeping all locals in the same scope alive until the next GC happens.
Today, these locals would go out of scope. There is lots of code which assumes that “local” resources – particularly open files – will be closed quickly. If closure has to wait for the next GC, a program (which runs fine today) may run out of file handles.
Making the __traceback__
attribute a weak reference would avoid the
problems with cyclic garbage. Unfortunately, it would make saving the
Exception
for later (as unittest
does) more awkward, and it would not
allow as much cleanup of the sys
module.
A possible alternate solution, suggested by Adam Olsen, would be to instead
turn the reference from the stack frame to the err
variable into a weak
reference when the variable goes out of scope [13].
Possible Future Compatible Changes
These changes are consistent with the appearance of exceptions as a single object rather than a triple at the interpreter level.
- If PEP 340 or PEP 343 is accepted, replace the three (
type
,value
,traceback
) arguments to__exit__
with a single exception argument. - Deprecate
sys.exc_type
,sys.exc_value
,sys.exc_traceback
, andsys.exc_info()
in favour of a single member,sys.exception
. - Deprecate
sys.last_type
,sys.last_value
, andsys.last_traceback
in favour of a single member,sys.last_exception
. - Deprecate the three-argument form of the
raise
statement in favour of the one-argument form. - Upgrade
cgitb.html()
to accept a single value as its first argument as an alternative to a(type, value, traceback)
tuple.
Possible Future Incompatible Changes
These changes might be worth considering for Python 3000.
- Remove
sys.exc_type
,sys.exc_value
,sys.exc_traceback
, andsys.exc_info()
. - Remove
sys.last_type
,sys.last_value
, andsys.last_traceback
. - Replace the three-argument
sys.excepthook
with a one-argument API, and changing thecgitb
module to match. - Remove the three-argument form of the
raise
statement. - Upgrade
traceback.print_exception
to accept anexception
argument instead of thetype
,value
, andtraceback
arguments.
Acknowledgements
Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip J. Eby, Raymond Hettinger, Walter Dörwald, and others.
References
Copyright
This document has been placed in the public domain.
Source: https://github.com/python/peps/blob/main/peps/pep-0344.rst
Last modified: 2023-09-09 17:39:29 GMT