[Twisted-Python] Tracebacks being dropped from exceptions when using inlineCallbacks

glyph at divmod.com glyph at divmod.com
Sat Jan 17 02:54:45 EST 2009

On 02:31 am, radix at twistedmatrix.com wrote:
>On Fri, Jan 16, 2009 at 7:28 PM, Phil Christensen 
><phil at bubblehouse.org> wrote:

>Keeping references to tracebacks still has many potential pitfalls.
>It's a fundamental problem: tracebacks refer to all their frames,
>which refer to all their locals; this makes it *really* easy to create
>uncollectable cycles if you have any __del__ methods *anywhere*. It's
>just not safe to keep the traceback reference in the general case.
>Failure needs to work with arbitrary code.

This used to be pretty much 100% of the time, since Deferred itself used 
to have a __del__, so if you had a Deferred anywhere on the stack you 
were dead.  Nowadays it's a little better than that, but a surprising 
number of objects from elsewhere are uncollectable.
>>Originally all __getstate__ did was stringify the object's state, 
>>which is
>>probably why cleanFailure calls it directly. It looks to me like most 
>>of the
>>code from __getstate__ should really be moved into a separate method, 
>>would be called from cleanFailure as well as __getstate__.

Nevertheless cleanFailure will still need to remove the traceback, for 
the reasons that Chris mentioned.
>Sure, that would be a slightly better factoring.

Would that help, though?

I don't fully understand the issue, so this is probably wrong in some 
obvious way, but: do we really need to hang on to a real traceback 
object to resolve the issue in question?  It sounds like the real issue 
here is that we don't have a channel to communicate a "cleaned 
traceback" (i.e., failure with stringified frames, but no traceback) 
from one bit of code to another?
>>>Note Beginning with Python 2.2, such cycles are automatically 
>>>when garbage collection is enabled and they become unreachable, but 
>>>remains more efficient to avoid creating cycles.

I'm not sure what Python documentation you're quoting here, but that's 
not accurate in the general case.  For example, see 

    "Objects that have __del__() methods and are part of a reference 
cycle     cause the entire reference cycle to be uncollectable".
>>I'd appreciate any feedback.

Two bits of feedback:

1. More of this discussion should be on a ticket.  The mailing list is 
good for getting discussions started and figuring out what needs to be 
done, but now we're talking about the technical specifics of resolving a 
specific problem.  Someone (terry?) should take the specifics in the 
email that started this thread and file an appropriate ticket.

2. When filing that ticket, I'd really like a full working example of 
inlineCallbacks not showing a traceback to play with, not just examples 
of portions of the problem.  The obvious example I tried,

    from twisted.internet.defer import inlineCallbacks

    def buggy2():
        raise RuntimeError("Whoops!")

    def buggy():
        yield buggy2()


actually prints a nice descriptive traceback.

More information about the Twisted-Python mailing list