id	summary	reporter	owner	description	type	status	priority	milestone	component	resolution	keywords	cc	branch	branch_author	launchpad_bug
3622	Traceback info removed from failure when inlineCallbacks decorated func calls deferred returning func that fails	terrycojones		"In the following code, a function (F1), decorated with defer.inlineCallbacks, calls another function (F2) that returns a deferred. A failure, containing a correct traceback, is passed to the errback of this deferred. However, due to the workings of the deferred failure processing (details below), the traceback in the exception information thrown back to F1 is None. So when F1 raises an exception based on this, the traceback (which is freshly generated) in the exception  shows the error as originating on the yield line instead of at the illegal division.

Here's the code:

{{{
#!python
from twisted.internet import reactor, defer

def ok(r): print 'OK', r
def nok(r): print 'FAILED', r

def F2():
    try:
        1 / 0
    except Exception:
        return defer.fail()

@defer.inlineCallbacks
def F1():
    yield F2()

def F0():
    F1().addCallbacks(ok, nok).addBoth(lambda _: reactor.stop())

reactor.callLater(0, F0)
reactor.run()
}}}

This produces the following output:

{{{
FAILED [Failure instance: Traceback: <type 'exceptions.ZeroDivisionError'>: integer division or modulo by zero
/usr/lib/python2.5/site-packages/twisted/internet/posixbase.py:228:mainLoop
/usr/lib/python2.5/site-packages/twisted/internet/base.py:561:runUntilCurrent
five.py:18:F0
/usr/lib/python2.5/site-packages/twisted/internet/defer.py:813:unwindGenerator
--- <exception caught here> ---
/usr/lib/python2.5/site-packages/twisted/internet/defer.py:724:_inlineCallbacks
five.py:15:F1
]
}}}

As you can see, the traceback shows the exception as having been raised in F1, not F2.

In more detail, the sequence of events is as below. If it's not clear what code I'm talking about, it's almost all in twisted.internet.defer either in _inlineCallbacks or in the Deferred class.

 1. When Python reads the input file and comes across F1, it creates F1d, an inlineCallbacks decorated version of F1.
 1. F0 calls F1 (really F1d).
 1. This results in _inlineCallbacks being called with the result of calling F1. Because F1 yields, _inlineCallbacks gets passed a generator G.
 1. _inlineCallbacks starts G, sending it None.
 1. G runs, calling F2.
 1. defer.fail is called in F2. This
    * creates a deferred and a failure and passes the failure it to errback on the deferred.
    * errback calls _startRunCallbacks with the failure.
    * _startRunCallbacks calls _runCallbacks.
    * There are no callbacks on the deferred.
    * Its result is a failure instance, so cleanFailure is called on it.
    * The failure has its tb attribute set to None by {{{__getstate__}}}.
 1. As a result, the deferred returned by F2 and received by F1d has a failure result in it with None in the tb.
 1. This deferred is received in G.
 1. gotResult is added to the deferred and is called immediately because the deferred has been called. gotResult puts the failure result into waiting{{{[1]}}} and this is then put into result.
 1. _inlineCallbacks loops and its result is now a failure.
 1. It throws the type, value, and (None) tb back to G.
 1. G raises the exception. The traceback is None, so a new one is created at yield in F1. Note: the exception still has type {{{ZeroDivisionError}}}
 1. This is caught by _inlineCallbacks which calls errback on the deferred F0 will receive.
 1. F0 adds its call and errback, which are called with the failure result.


In summary, the failure that propagates back to the nok method is of type {{{ZeroDivisionError}}} (corresponding to the original exception) but its traceback is created by the exception that's raised when F1 yields F2(). The original traceback is gone.

As a result, the programmer who has used inlineCallbacks knows the type of error that has occurred but has no further information about where it happened. This can of course be complicated to track down if F2 is non-trivial and calls other functions (some of which might in turn use inlineCallbacks which might in turn run into the same problem).

Apologies for so much detail. Maybe it's not necessary.
"	defect	closed	normal		core	invalid	inlineCallbacks tracebacks