python / peps

Sponsor

Sponsor python/peps
Notifications
Star 2.7k
Fork 1k

Cannot retrieve contributors at this time

578 lines (420 sloc) 21 KB

	PEP: 3134
	Title: Exception Chaining and Embedded Tracebacks
	Version: $Revision$
	Last-Modified: $Date$
	Author: Ka-Ping Yee
	Status: Final
	Type: Standards Track
	Content-Type: text/x-rst
	Created: 12-May-2005
	Python-Version: 3.0
	Post-History:


	Numbering Note
	==============

	This PEP started its life as PEP 344. Since it is now targeted for Python
	3000, it has been moved into the 3xxx space.


	Abstract
	========

	This PEP proposes three standard attributes on exception instances: the
	``__context__`` attribute for implicitly chained exceptions, the ``__cause__``
	attribute for explicitly chained exceptions, and the ``__traceback__``
	attribute for the traceback. A new ``raise ... from`` statement sets the
	``__cause__`` attribute.


	Motivation
	==========

	During the handling of one exception (exception A), it is possible that another
	exception (exception B) may occur. In today's Python (version 2.4), if this
	happens, exception B is propagated outward and exception A is lost. In order
	to debug the problem, it is useful to know about both exceptions. The
	``__context__`` attribute retains this information automatically.

	Sometimes it can be useful for an exception handler to intentionally re-raise
	an exception, either to provide extra information or to translate an exception
	to another type. The ``__cause__`` attribute provides an explicit way to
	record the direct cause of an exception.

	In today's Python implementation, exceptions are composed of three parts: the
	type, the value, and the traceback. The ``sys`` module, exposes the current
	exception in three parallel variables, ``exc_type``, ``exc_value``, and
	``exc_traceback``, the ``sys.exc_info()`` function returns a tuple of these
	three parts, and the ``raise`` statement has a three-argument form accepting
	these three parts. Manipulating exceptions often requires passing these three
	things in parallel, which can be tedious and error-prone. Additionally, the
	``except`` statement can only provide access to the value, not the traceback.
	Adding the ``__traceback__`` attribute to exception values makes all the
	exception information accessible from a single place.


	History
	=======

	Raymond Hettinger [1]_ raised the issue of masked exceptions on Python-Dev in
	January 2003 and proposed a ``PyErr_FormatAppend()`` function that C modules
	could use to augment the currently active exception with more information.
	Brett Cannon [2]_ brought up chained exceptions again in June 2003, prompting
	a long discussion.

	Greg Ewing [3]_ identified the case of an exception occurring in a ``finally``
	block during unwinding triggered by an original exception, as distinct from
	the case of an exception occurring in an ``except`` block that is handling the
	original exception.

	Greg Ewing [4]_ and Guido van Rossum [5]_, and probably others, have
	previously mentioned adding a traceback attribute to Exception instances.
	This is noted in PEP 3000.

	This PEP was motivated by yet another recent Python-Dev reposting of the same
	ideas [6]_ [7]_.


	Rationale
	=========

	The Python-Dev discussions revealed interest in exception chaining for two
	quite different purposes. To handle the unexpected raising of a secondary
	exception, the exception must be retained implicitly. To support intentional
	translation of an exception, there must be a way to chain exceptions
	explicitly. This PEP addresses both.

	Several attribute names for chained exceptions have been suggested on
	Python-Dev [2]_, including ``cause``, ``antecedent``, ``reason``, ``original``,
	``chain``, ``chainedexc``, ``exc_chain``, ``excprev``, ``previous``, and
	``precursor``. For an explicitly chained exception, this PEP suggests
	``__cause__`` because of its specific meaning. For an implicitly chained
	exception, this PEP proposes the name ``__context__`` because the intended
	meaning is more specific than temporal precedence but less specific than
	causation: an exception occurs in the context of handling another exception.

	This PEP suggests names with leading and trailing double-underscores for these
	three attributes because they are set by the Python VM. Only in very special
	cases should they be set by normal assignment.

	This PEP handles exceptions that occur during ``except`` blocks and ``finally``
	blocks in the same way. Reading the traceback makes it clear where the
	exceptions occurred, so additional mechanisms for distinguishing the two cases
	would only add unnecessary complexity.

	This PEP proposes that the outermost exception object (the one exposed for
	matching by ``except`` clauses) be the most recently raised exception for
	compatibility with current behaviour.

	This PEP proposes that tracebacks display the outermost exception last, because
	this would be consistent with the chronological order of tracebacks (from
	oldest to most recent frame) and because the actual thrown exception is easier
	to find on the last line.

	To keep things simpler, the C API calls for setting an exception will not
	automatically set the exception's ``__context__``. Guido van Rossum has
	expressed concerns with making such changes [8]_.

	As for other languages, Java and Ruby both discard the original exception when
	another exception occurs in a ``catch``/``rescue`` or ``finally``/``ensure``
	clause. Perl 5 lacks built-in structured exception handling. For Perl 6, RFC
	number 88 [9]_ proposes an exception mechanism that implicitly retains chained
	exceptions in an array named ``@@``. In that RFC, the most recently raised
	exception is exposed for matching, as in this PEP; also, arbitrary expressions
	(possibly involving ``@@``) can be evaluated for exception matching.

	Exceptions in C# contain a read-only ``InnerException`` property that may point
	to another exception. Its documentation [10]_ says that "When an exception X
	is thrown as a direct result of a previous exception Y, the ``InnerException``
	property of X should contain a reference to Y." This property is not set by
	the VM automatically; rather, all exception constructors take an optional
	``innerException`` argument to set it explicitly. The ``__cause__`` attribute
	fulfills the same purpose as ``InnerException``, but this PEP proposes a new
	form of ``raise`` rather than extending the constructors of all exceptions. C#
	also provides a ``GetBaseException`` method that jumps directly to the end of
	the ``InnerException`` chain; this PEP proposes no analog.

	The reason all three of these attributes are presented together in one proposal
	is that the ``__traceback__`` attribute provides convenient access to the
	traceback on chained exceptions.


	Implicit Exception Chaining
	===========================

	Here is an example to illustrate the ``__context__`` attribute::

	def compute(a, b):
	try:
	a/b
	except Exception, exc:
	log(exc)

	def log(exc):
	file = open('logfile.txt') # oops, forgot the 'w'
	print >>file, exc
	file.close()

	Calling ``compute(0, 0)`` causes a ``ZeroDivisionError``. The ``compute()``
	function catches this exception and calls ``log(exc)``, but the ``log()``
	function also raises an exception when it tries to write to a file that wasn't
	opened for writing.

	In today's Python, the caller of ``compute()`` gets thrown an ``IOError``. The
	``ZeroDivisionError`` is lost. With the proposed change, the instance of
	``IOError`` has an additional ``__context__`` attribute that retains the
	``ZeroDivisionError``.

	The following more elaborate example demonstrates the handling of a mixture of
	``finally`` and ``except`` clauses::

	def main(filename):
	file = open(filename) # oops, forgot the 'w'
	try:
	try:
	compute()
	except Exception, exc:
	log(file, exc)
	finally:
	file.clos() # oops, misspelled 'close'

	def compute():
	1/0

	def log(file, exc):
	try:
	print >>file, exc # oops, file is not writable
	except:
	display(exc)

	def display(exc):
	print ex # oops, misspelled 'exc'

	Calling ``main()`` with the name of an existing file will trigger four
	exceptions. The ultimate result will be an ``AttributeError`` due to the
	misspelling of ``clos``, whose ``__context__`` points to a ``NameError`` due
	to the misspelling of ``ex``, whose ``__context__`` points to an ``IOError``
	due to the file being read-only, whose ``__context__`` points to a
	``ZeroDivisionError``, whose ``__context__`` attribute is ``None``.

	The proposed semantics are as follows:

	1. Each thread has an exception context initially set to ``None``.

	2. Whenever an exception is raised, if the exception instance does not already
	have a ``__context__`` attribute, the interpreter sets it equal to the
	thread's exception context.

	3. Immediately after an exception is raised, the thread's exception context is
	set to the exception.

	4. Whenever the interpreter exits an ``except`` block by reaching the end or
	executing a ``return``, ``yield``, ``continue``, or ``break`` statement, the
	thread's exception context is set to ``None``.


	Explicit Exception Chaining
	===========================

	The ``__cause__`` attribute on exception objects is always initialized to
	``None``. It is set by a new form of the ``raise`` statement::

	raise EXCEPTION from CAUSE

	which is equivalent to::

	exc = EXCEPTION
	exc.__cause__ = CAUSE
	raise exc

	In the following example, a database provides implementations for a few
	different kinds of storage, with file storage as one kind. The database
	designer wants errors to propagate as ``DatabaseError`` objects so that the
	client doesn't have to be aware of the storage-specific details, but doesn't
	want to lose the underlying error information.

	::

	class DatabaseError(Exception):
	pass

	class FileDatabase(Database):
	def __init__(self, filename):
	try:
	self.file = open(filename)
	except IOError, exc:
	raise DatabaseError('failed to open') from exc

	If the call to ``open()`` raises an exception, the problem will be reported as
	a ``DatabaseError``, with a ``__cause__`` attribute that reveals the
	``IOError`` as the original cause.


	Traceback Attribute
	===================

	The following example illustrates the ``__traceback__`` attribute.

	::

	def do_logged(file, work):
	try:
	work()
	except Exception, exc:
	write_exception(file, exc)
	raise exc

	from traceback import format_tb

	def write_exception(file, exc):
	...
	type = exc.__class__
	message = str(exc)
	lines = format_tb(exc.__traceback__)
	file.write(... type ... message ... lines ...)
	...

	In today's Python, the ``do_logged()`` function would have to extract the
	traceback from ``sys.exc_traceback`` or ``sys.exc_info()`` [2]_ and pass both
	the value and the traceback to ``write_exception()``. With the proposed
	change, ``write_exception()`` simply gets one argument and obtains the
	exception using the ``__traceback__`` attribute.

	The proposed semantics are as follows:

	1. Whenever an exception is caught, if the exception instance does not already
	have a ``__traceback__`` attribute, the interpreter sets it to the newly
	caught traceback.


	Enhanced Reporting
	==================

	The default exception handler will be modified to report chained exceptions.
	The chain of exceptions is traversed by following the ``__cause__`` and
	``__context__`` attributes, with ``__cause__`` taking priority. In keeping
	with the chronological order of tracebacks, the most recently raised exception
	is displayed last; that is, the display begins with the description of the
	innermost exception and backs up the chain to the outermost exception. The
	tracebacks are formatted as usual, with one of the lines::

	The above exception was the direct cause of the following exception:

	or

	::

	During handling of the above exception, another exception occurred:

	between tracebacks, depending whether they are linked by ``__cause__`` or
	``__context__`` respectively. Here is a sketch of the procedure::

	def print_chain(exc):
	if exc.__cause__:
	print_chain(exc.__cause__)
	print '\nThe above exception was the direct cause...'
	elif exc.__context__:
	print_chain(exc.__context__)
	print '\nDuring handling of the above exception, ...'
	print_exc(exc)

	In the ``traceback`` module, the ``format_exception``, ``print_exception``,
	``print_exc``, and ``print_last`` functions will be updated to accept an
	optional ``chain`` argument, ``True`` by default. When this argument is
	``True``, these functions will format or display the entire chain of exceptions
	as just described. When it is ``False``, these functions will format or
	display only the outermost exception.

	The ``cgitb`` module should also be updated to display the entire chain of
	exceptions.


	C API
	=====

	The ``PyErr_Set*`` calls for setting exceptions will not set the
	``__context__`` attribute on exceptions. ``PyErr_NormalizeException`` will
	always set the ``traceback`` attribute to its ``tb`` argument and the
	``__context__`` and ``__cause__`` attributes to ``None``.

	A new API function, ``PyErr_SetContext(context)``, will help C programmers
	provide chained exception information. This function will first normalize the
	current exception so it is an instance, then set its ``__context__`` attribute.
	A similar API function, ``PyErr_SetCause(cause)``, will set the ``__cause__``
	attribute.


	Compatibility
	=============

	Chained exceptions expose the type of the most recent exception, so they will
	still match the same ``except`` clauses as they do now.

	The proposed changes should not break any code unless it sets or uses
	attributes named ``__context__``, ``__cause__``, or ``__traceback__`` on
	exception instances. As of 2005-05-12, the Python standard library contains no
	mention of such attributes.


	Open Issue: Extra Information
	==============================

	Walter Dörwald [11]_ expressed a desire to attach extra information to an
	exception during its upward propagation without changing its type. This could
	be a useful feature, but it is not addressed by this PEP. It could conceivably
	be addressed by a separate PEP establishing conventions for other informational
	attributes on exceptions.


	Open Issue: Suppressing Context
	================================

	As written, this PEP makes it impossible to suppress ``__context__``, since
	setting ``exc.__context__`` to ``None`` in an ``except`` or ``finally`` clause
	will only result in it being set again when ``exc`` is raised.


	Open Issue: Limiting Exception Types
	=====================================

	To improve encapsulation, library implementors may want to wrap all
	implementation-level exceptions with an application-level exception. One could
	try to wrap exceptions by writing this::

	try:
	... implementation may raise an exception ...
	except:
	import sys
	raise ApplicationError from sys.exc_value

	or this::

	try:
	... implementation may raise an exception ...
	except Exception, exc:
	raise ApplicationError from exc

	but both are somewhat flawed. It would be nice to be able to name the current
	exception in a catch-all ``except`` clause, but that isn't addressed here.
	Such a feature would allow something like this::

	try:
	... implementation may raise an exception ...
	except *, exc:
	raise ApplicationError from exc


	Open Issue: yield
	==================

	The exception context is lost when a ``yield`` statement is executed; resuming
	the frame after the ``yield`` does not restore the context. Addressing this
	problem is out of the scope of this PEP; it is not a new problem, as
	demonstrated by the following example::

	>>> def gen():
	... try:
	... 1/0
	... except:
	... yield 3
	... raise
	...
	>>> g = gen()
	>>> g.next()
	3
	>>> g.next()
	TypeError: exceptions must be classes, instances, or strings
	(deprecated), not NoneType


	Open Issue: Garbage Collection
	===============================

	The strongest objection to this proposal has been that it creates cycles
	between exceptions and stack frames [12]_. Collection of cyclic garbage (and
	therefore resource release) can be greatly delayed.

	::

	>>> try:
	>>> 1/0
	>>> except Exception, err:
	>>> pass

	will introduce a cycle from err -> traceback -> stack frame -> err, keeping all
	locals in the same scope alive until the next GC happens.

	Today, these locals would go out of scope. There is lots of code which assumes
	that "local" resources -- particularly open files -- will be closed quickly.
	If closure has to wait for the next GC, a program (which runs fine today) may
	run out of file handles.

	Making the ``__traceback__`` attribute a weak reference would avoid the
	problems with cyclic garbage. Unfortunately, it would make saving the
	``Exception`` for later (as ``unittest`` does) more awkward, and it would not
	allow as much cleanup of the ``sys`` module.

	A possible alternate solution, suggested by Adam Olsen, would be to instead
	turn the reference from the stack frame to the ``err`` variable into a weak
	reference when the variable goes out of scope [13]_.


	Possible Future Compatible Changes
	==================================

	These changes are consistent with the appearance of exceptions as a single
	object rather than a triple at the interpreter level.

	- If PEP 340 or PEP 343 is accepted, replace the three (``type``, ``value``,
	``traceback``) arguments to ``__exit__`` with a single exception argument.

	- Deprecate ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and
	``sys.exc_info()`` in favour of a single member, ``sys.exception``.

	- Deprecate ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback``
	in favour of a single member, ``sys.last_exception``.

	- Deprecate the three-argument form of the ``raise`` statement in favour of the
	one-argument form.

	- Upgrade ``cgitb.html()`` to accept a single value as its first argument as an
	alternative to a ``(type, value, traceback)`` tuple.


	Possible Future Incompatible Changes
	====================================

	These changes might be worth considering for Python 3000.

	- Remove ``sys.exc_type``, ``sys.exc_value``, ``sys.exc_traceback``, and
	``sys.exc_info()``.

	- Remove ``sys.last_type``, ``sys.last_value``, and ``sys.last_traceback``.

	- Replace the three-argument ``sys.excepthook`` with a one-argument API, and
	changing the ``cgitb`` module to match.

	- Remove the three-argument form of the ``raise`` statement.

	- Upgrade ``traceback.print_exception`` to accept an ``exception`` argument
	instead of the ``type``, ``value``, and ``traceback`` arguments.


	Implementation
	==============

	The ``__traceback__`` and ``__cause__`` attributes and the new raise syntax
	were implemented in revision 57783 [14]_.


	Acknowledgements
	================

	Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip J. Eby,
	Raymond Hettinger, Walter Dörwald, and others.


	References
	==========

	.. [1] Raymond Hettinger, "Idea for avoiding exception masking"
	https://mail.python.org/pipermail/python-dev/2003-January/032492.html

	.. [2] Brett Cannon explains chained exceptions
	https://mail.python.org/pipermail/python-dev/2003-June/036063.html

	.. [3] Greg Ewing points out masking caused by exceptions during finally
	https://mail.python.org/pipermail/python-dev/2003-June/036290.html

	.. [4] Greg Ewing suggests storing the traceback in the exception object
	https://mail.python.org/pipermail/python-dev/2003-June/036092.html

	.. [5] Guido van Rossum mentions exceptions having a traceback attribute
	https://mail.python.org/pipermail/python-dev/2005-April/053060.html

	.. [6] Ka-Ping Yee, "Tidier Exceptions"
	https://mail.python.org/pipermail/python-dev/2005-May/053671.html

	.. [7] Ka-Ping Yee, "Chained Exceptions"
	https://mail.python.org/pipermail/python-dev/2005-May/053672.html

	.. [8] Guido van Rossum discusses automatic chaining in ``PyErr_Set*``
	https://mail.python.org/pipermail/python-dev/2003-June/036180.html

	.. [9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism"
	http://dev.perl.org/perl6/rfc/88.html

	.. [10] MSDN .NET Framework Library, "Exception.InnerException Property"
	http://msdn.microsoft.com/library/en-us/cpref/html/frlrfsystemexceptionclassinnerexceptiontopic.asp

	.. [11] Walter Dörwald suggests wrapping exceptions to add details
	https://mail.python.org/pipermail/python-dev/2003-June/036148.html

	.. [12] Guido van Rossum restates the objection to cyclic trash
	https://mail.python.org/pipermail/python-3000/2007-January/005322.html

	.. [13] Adam Olsen suggests using a weakref from stack frame to exception
	https://mail.python.org/pipermail/python-3000/2007-January/005363.html

	.. [14] Patch to implement the bulk of the PEP
	http://svn.python.org/view/python/branches/py3k/Include/?rev=57783&view=rev



	Copyright
	=========

	This document has been placed in the public domain.


	..
	Local Variables:
	mode: indented-text
	indent-tabs-mode: nil
	sentence-end-double-space: t
	fill-column: 70
	coding: utf-8
	End:

python / peps

Sponsor python/peps

peps/pep-3134.txt