[Twisted-Python] how to write a safe catch-all
p.mayers at imperial.ac.uk
Thu Sep 30 08:44:42 EDT 2010
On 30/09/10 11:48, Chris Withers wrote:
> On 30/09/2010 04:45, exarkun at twistedmatrix.com wrote:
>> On 12:52 am, chris at simplistix.co.uk wrote:
>>> Because I haven't found any permutation that doesn't result in the
>>> LoopingCall's calls to `loop` from stopping after the exception.
>>> I would be more than ecstatic to be proved wrong ;-)
>> You keep saying the LoopingCall stops. It does not stop. It is waiting
>> for a result.
> What result is it waiting for and how do I pass that from the `loop`
> function in my first example when the exception is caught?
It is waiting for the deferred you return to callback/errback.
This, I found, was initially non-obvious. I assumed that LoopingCall
would be called every N seconds regardless, but it won't. If you return
a deferred, the LoopingCall will stop until the deferred
callback/errback's, then it will be restarted.
This is very useful in some circumstances.
>> If you provide it with a result, it will proceed.
> See above: how do I do that?
>> suggested a number of (hackfully) mechanisms you might use to provide it
>> with a result.
> Why do you say hackfully? I'm trying to build a scheduler that is
Because this is (apparently) a bug in the underlying Twisted code, so
anything you do to fix that is a hack.
The quickest way I can describe it is this; as I understand, any Twisted
function which returns a deferred should never raise an Exception. The
exception should instead flow down the errback.
This is because (as you've discovered) there are circumstances when a
function is called in Twisted where the exception can't propagate "up"
to the caller, because the caller is no longer higher in the call stack.
Most prominent is reactor.callLater.
> resilient to failure in a scheduled task. Yes, of course I want to fix
> that failure, but I don't want a failure in one scheduled task to
> prevent any further scheduled tasks from being executed...
>> I suggested another way (fix the broken code, or at
>> least supply enough information for someone else to fix it).
> For all I know, it's already fixed in a newer version of Twisted
> (unfortunately, I can't move this project to a newer version of Twisted)
> but once I have a robust scheduler, I'll certainly do my best to report
> the underlying failure to you guys properly...
If you can produce a short example that uses the real Twisted API which
fails, people might a) be able to find/fix the bug and b) suggest a less
hacky way of ensuring you see the error.
More information about the Twisted-Python