[Twisted-Python] If the errbacks of a canceled Deferred are called with error other than CancelledError, is this acceptable?

[Terry Jones]

Hi Tom

I'm not sure we should continue this discussion on the list seeing as the
original issue seems to be settled. Anyway, here's an answer and maybe we
could/should take it offline if there's more to say?

On Sat, Jun 22, 2013 at 11:14 PM, Tom Prince <tom.prince at ualberta.net>wrote:

> Terry Jones <terry at jon.es> writes:
> > The problem that I was originally trying to address by getting 'cancel'
> > into deferreds is that the control flow you mention gets totally stuck
> if a
> > deferred never fires for some reason (this was happening to me in talking
> > to the Twitter API).
>
> .cancel as currently implemented addresses this, as calling cancel
> causes the deferred to fire.


Right. When I say "originally" I mean in Jan 2010, before cancel was
implemented. See the thread URL I posted a couple of times in this thread
to see how it wen t down.


> > This goes against the thinking that only the code that creates a deferred
> > can fire it.
>
> To allow anything else would be to break the abstraction
>
>     d = Deferred()
>     return d
>     ...
>     d.callback(5)
>
> vs.
>     d = Deferred()
>     d.addCallback(lambda x: x+1)
>     return d
>     ...
>     d.callback(4)
>
> There is currently no way of distinguishing these to code
> snippets. Allowing other code to callback/errback the deferred
> would cause different results in the two cases.


That depends how you do it. If you actually have 2 deferreds, the original
(which you can't err/callback yourself cause things will break) and one
that you create yourself (as a "controllable" deferred), then you can allow
the latter to be call/errbacked and hook the former up to the latter if the
former fires first. That's what the code samples I posted do, including the
one from Jan 2010.


> > As API writers, we're used to being in control: we don't think it's a
> > good idea if we create and return a deferred in a method and the code
> > we return the deferred to then fires it itself.
>
> It isn't so much a matter of control, as of abstraction. It isn't
> uncommon for a single deferred to be pass through several systems, with
> each stage adding more callbacks, changing the value that the next stage
> will see in the callbacks they add.
>

Right.


> If the "final" consumer could fire the deferred with any value, it would
> need to know about all the stagse (or at least the first one) to know
> what kind of value to put in.


In the ControlableDeferred2013 class I posted the other night, this isn't
the case. If the deferred using code fires it, the only callbacks and
errbacks that are run are ones that that code added (not ones added to the
original deferred by the code that made it or any intermediates).


> > But as app writers consuming deferreds from these APIs, we
> > want more control (I do, anyway).
>
> Well, the consumer can't control the deferreds that it gets but, as your
> code demonstrates, it is easy enough to create a new deferred that
> behaves however the consumer wants. The fact that this isn't the same
> object isn't signifiant, since the identity of deferreds isn't an
> interesting thing.


Yes.

> The ControllableDeferred2013 class I posted last night shows one
> > way.
>
> One comment about the code you posted: There doesn't really seem to be a
> need to create a sepearte class, that forwards
> .addCallback/.addErrback. You can just create a regular deferred that is
> hooked hooked up in the appropriate way. (It may make sense for the
> stage that sets this up to have an object, but it shouldn't pass it
> along, it should just pass a regular deferred on). There are a few
> places that accept deferreds that depend on the object being an instance
> of deferred, which your object isn't.


Yes, you can do that too (and there are other approaches). Here's a
subclassing version that's similar to what you just described:
http://twistedmatrix.com/pipermail/twisted-python/2010-January/021311.html
They're all basically the same idea (at least the way I do it), just based
on setting up another deferred and hooking the original up to it when/if
needed.

Terry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130623/e2d70cf3/attachment.html>