[Twisted-Python] The Real Error (tm) [was Re: how to write a safe catch-all]

[Phil Mayers]

On 30/09/10 16:10, Chris Withers wrote:
> On 30/09/2010 15:53, Phil Mayers wrote:
>>> So, I appear to be back to the case where I can either gracefully handle
>>> the exception *or* gracefully handle the errback, but not both?
>>
>> It should be possible if using Twisted APIs correctly to reliably
>> capture errors.
>>
>> If it is not, that is a Twisted bug (or possibly API limitation)
>>
>> However, your "test_looping.py" is not using the API correctly; the
>> reactor.callLater throws away the call stack, so the error has nowhere
>> to go.
>
> Right, but that's what appears to be happening with the "real code", and
> I get the mysterious GeneratorExit...

It is more than a little confusing, and I'm sure frustrating.

I've tried to produce something like this locally, but cannot. I suspect 
the various @inlineCallbacks generators are the cause of the specific issue.


Let me see if I understand the problem in full.

You have an @inlineCallbacks-decorated generator which is the target of 
a LoopingCall, like so:

@inlineCallbacks
def loop():
   try:
     somework()
   except:
     log.err()

lc = task.LoopingCall(loop)

You want this loop function to catch & log all exceptions resulting from 
work it initiates.

Your "somework" function calls, amongst other things, an 
@inlineCallbacks-decorated worker function:

@inlineCallbacks
def sendTransmission(...):
   try:
     yield maybeDeferred(feed.initiateTransmission)
   except:
     ...some handling

You are seeing two errors:


  1. A GeneratorExit exception. This appears (if I'm reading your 
logging right) to be caught by your logging?

  2. A ConnectionLost exception. This is not caught by your logging, and 
is propagating up to the reactor, and giving "Unhandled Error"


Is this an accurate summary?