throwExceptionIntoGenerator sometimes swallows traceback

[alanfranzoni]

This is a bit tricky; I found this behaviour in some pretty complex code, and it was not so easy to provide a test case. I realize it is a corner case (and it can be worked around in application code) but I think it is correct to report it.

Basically: if an exception is raised by a generator, gets wrapped in a failure and then the same exception is rethrown inside the same generator, all the frames that *really* matter (e.g. where the exception occurs) aren't shown in the failure output.

gen_throw_fail.py output:

(myenv)AlanMbPro:twisted_gen alan$ python2.6 gen_throw_fail.py 
Unhandled error in Deferred:
Traceback (most recent call last):
  File "/Users/alan/Dropbox/code/Twisted-bzr/trunk/myenv/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-macosx-10.5-i386.egg/twisted/internet/base.py", line 409, in _continueFiring
    callable(*args, **kwargs)
  File "gen_throw_fail.py", line 30, in start
    d.addErrback(handleError, g2)
  File "/Users/alan/Dropbox/code/Twisted-bzr/trunk/myenv/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-macosx-10.5-i386.egg/twisted/internet/defer.py", line 200, in addErrback
    errbackKeywords=kw)
  File "/Users/alan/Dropbox/code/Twisted-bzr/trunk/myenv/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-macosx-10.5-i386.egg/twisted/internet/defer.py", line 182, in addCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/Users/alan/Dropbox/code/Twisted-bzr/trunk/myenv/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-macosx-10.5-i386.egg/twisted/internet/defer.py", line 324, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "gen_throw_fail.py", line 18, in handleError
    f.throwExceptionIntoGenerator(g)
  File "/Users/alan/Dropbox/code/Twisted-bzr/trunk/myenv/lib/python2.6/site-packages/Twisted-9.0.0-py2.6-macosx-10.5-i386.egg/twisted/python/failure.py", line 338, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
exceptions.ValueError: some error

It can be seen quite clearly; the exception name and message is shown, but the line where it occurs is not.

Normal python behaviour is to add a frame at the bottom of the stack, but all traceback is preserved:

python_gen_errors.py output:

(myenv)AlanMbPro:twisted_gen alan$ python2.6 python_gen_errors.py 
Normal traceback:
Traceback (most recent call last):
  File "python_gen_errors.py", line 19, in <module>
    g.send(None)
  File "python_gen_errors.py", line 10, in gen
    raise ValueError, "this is an error"
ValueError: this is an error

Rethrown traceback:
Traceback (most recent call last):
  File "python_gen_errors.py", line 31, in <module>
    g.throw(*sys.exc_info())
  File "python_gen_errors.py", line 28, in <module>
    g.send(None)
  File "python_gen_errors.py", line 10, in gen
    raise ValueError, "this is an error"
ValueError: this is an error

I tried some debugging, but this mingles heavily with some of twisted Failure internals and I don't seem to be able to cope with that.