[Twisted-Python] The Real Error (tm) [was Re: how to write a safe catch-all]

Phil Mayers p.mayers at imperial.ac.uk
Thu Sep 30 13:01:03 EDT 2010

On 30/09/10 16:10, Chris Withers wrote:
> On 30/09/2010 15:53, Phil Mayers wrote:
>>> So, I appear to be back to the case where I can either gracefully handle
>>> the exception *or* gracefully handle the errback, but not both?
>> It should be possible if using Twisted APIs correctly to reliably
>> capture errors.
>> If it is not, that is a Twisted bug (or possibly API limitation)
>> However, your "test_looping.py" is not using the API correctly; the
>> reactor.callLater throws away the call stack, so the error has nowhere
>> to go.
> Right, but that's what appears to be happening with the "real code", and
> I get the mysterious GeneratorExit...

It is more than a little confusing, and I'm sure frustrating.

I've tried to produce something like this locally, but cannot. I suspect 
the various @inlineCallbacks generators are the cause of the specific issue.

Let me see if I understand the problem in full.

You have an @inlineCallbacks-decorated generator which is the target of 
a LoopingCall, like so:

def loop():

lc = task.LoopingCall(loop)

You want this loop function to catch & log all exceptions resulting from 
work it initiates.

Your "somework" function calls, amongst other things, an 
@inlineCallbacks-decorated worker function:

def sendTransmission(...):
     yield maybeDeferred(feed.initiateTransmission)
     ...some handling

You are seeing two errors:

  1. A GeneratorExit exception. This appears (if I'm reading your 
logging right) to be caught by your logging?

  2. A ConnectionLost exception. This is not caught by your logging, and 
is propagating up to the reactor, and giving "Unhandled Error"

Is this an accurate summary?

More information about the Twisted-Python mailing list