[Twisted-Python] how to write a safe catch-all

Chris Withers chris at simplistix.co.uk
Thu Sep 30 10:02:04 EDT 2010



On 30/09/2010 14:46, Phil Mayers wrote:
> On 30/09/10 14:36, Chris Withers wrote:
>>
>> Indeed, but, as I keep saying, I need to build a scheduler that's
>> resilient to errors in the tasks its running ;-)
>
> Sure.
>
>>
>>>> Most prominent is reactor.callLater.
>>
>> ...which I'm not actually using, it was just a small-as-possible way I
>> could simulate the failure mode (rather than the specific failure) I
>> need to protect against.
>
> Ah.
>
>> Actually, what appears to work is simply changing `loop` to not be an
>> async fuction:
>>
>> def loop():
>> try:
>> doStuff()
>> except Exception,e:
>> log.err(None,'Unhandled scheduled exception')
>>
>> looper = task.LoopingCall(loop)
>> looper.start(1.0)
>> reactor.run()
>>
>> This appears to solve most of the problems:
>> - the schedule now keeps running regardless
>> - exceptions in doStuff and below get logged
>>
>> However, it now has the problem that if doStuff *does* return a deferred
>> and it errbacks, the I get the ugly:
>
> Well, you could do this, which is more Twist-y, and Glyph suggested:
>
> def loop():
> d = defer.maybeDeferred(doStuff, *args)
> d.addErrback(log.err)

Okay, but further down in the call stack of the "real app", I already have:

     @inlineCallbacks
     def sendTransmission(self...):
         ...
         try:
             yield maybeDeferred(feed.initiateTransmission,
                                 ...)
         except Exception, ex:
             if isinstance(ex,GeneratorExit):
                 raise
             ...

Is that not doing what you're talking about?
If it is, it didn't help...

> FWIW, you say "if doStuff does return a deferred"; I presume you don't
> really have a single call returning both normal and deferred values,

Correct, doStuff *should* always return a deferred, the problem is that 
the deferred returned never fires (errback or callback) when an 
exception is raised in the code that should be firing it.

Chris



More information about the Twisted-Python mailing list