[Twisted-Python] A pseudo-deferred class that can be canceled
terry at jon.es
Mon Jan 4 22:54:46 EST 2010
>>>>> "Glyph" == Glyph Lefkowitz <glyph at twistedmatrix.com> writes:
Glyph> I already lost you at the first sentence.
Glyph> The class below never appears to use 'self._f'
Oops, that should have been a self._f in the call method.
Glyph> and ... Deferreds are things that fire, I don't see how the callable
Glyph> (f) can fire. Can you rephrase your intent?
I wanted a way to be able to call something (anything) that returns a
deferred and then, at some later point, to be able to decide that I no
longer want the deferred to fire in the way the original function would
have done it. Instead I want to fire it myself, to in some sense cancel the
outstanding call (by ignoring its result and instead returning a result of
my choosing, right now).
E.g., I call twisted.web.client.getPage and get a deferred (d) back. I add
a callback to d. If sometime later (perhaps just due to too much time
elapsing, but perhaps for another reason - e.g., maybe I'm shutting down a
service, who knows) I want to effectively cancel that deferred, what choice
do I have? The deferred was made elsewhere (in HTTPClientFactory in this
case), which is presumably going to fire it at some point. Somewhere else
in my code I've now got a callback function that's going to get the result,
though I now don't want that to happen. I could set a flag somewhere to
indicate to my own callback(s) that it(they) should ignore any incoming
result and/or fail, but that's pretty messy. Instead I'd like to just
manually cancel the deferred myself - where by "cancel" I mean fire it with
a value of my choosing. Most convenient, probably, is to call its errback
with a TimeoutError of some kind (just as defer.timeout does) which I can
just absorb or log etc. I can't do this though with the deferred I got
back from getPage.
So the (untested) class I posted sits in the middle. If the original
deferred (from getPage, in our example) fires first, it passes (via
chainDeferred) the result along to the deferred it gave me. If instead I
trigger the deferred myself (by calling 'callback' or 'errback' *on the
CancellableDeferred instance*) then it fires the deferred it gave me back
and will later ignore the result from getPage (if any).
Glyph> I'm glad you're thinking about this, because it is an *extremely*
Glyph> thorny issue which I would really like to address one day. Many of
Glyph> the issues you're talking about were brought up, and various
Glyph> solutions suggested, then found problematic, then modified ad
Glyph> nauseam on <http://twistedmatrix.com/trac/ticket/990>. If you can
Glyph> read that discussion and make some sense of it, perhaps you can post
Glyph> a recommendation there, or at least a summary of the discussion so
Glyph> far so that I don't have to read the whole discussion again to
Glyph> remember what I think should happen next? :)
OK, I'll take a look (in my copious spare time, etc).
>> I don't know why defer.Deferred.setTimeout is deprecated, but I guess
>> it's partly to do with this control issue.
Glyph> This is one of the reasons, but another major reason is that
Glyph> 'setTimeout' does not belong as a method of Deferred. If we did
Glyph> support cancellation somehow, the way to set a timeout would be to
Glyph> do 'reactor.callLater(5.0, myDeferred.cancel)'. Deferred was
Glyph> originally in twisted.python and it really should have remained
Glyph> there, decoupled from twisted.internet.
OK, I agree with that aim. Cancelation (if you want to call it that) using
my class is, IMO, decoupled from normal deferreds. Under normal
circumstances you don't want to cancel a deferred you've asked for. But if
you might, you can get yourself an instance of CancellableDeferred and use
it instead. It provides you with a way to cancel the deferred (by which I
mean: manually trigger it yourself with a value of your choosing, and
thereafter ignore anything that might eventually come back from whatever it
was you originally called (and which it is presumably now too late to
undo)). You can do the canceling via reactor.callLater, just as you
describe, but you call a method on the CancellableDeferred instance, not on
the deferred. That keeps the canceling functionality out of the deferred
Is that any clearer? Sorry the code was untested - it was meant to be so
simple that it couldn't possibly be wrong :-)
More information about the Twisted-Python