[Twisted-Python] A few comments on deferreds

[Jean-Paul Calderone]

On Fri, 6 Mar 2009 12:10:55 +0100, Terry Jones <terry at jon.es> wrote:
>Here are a few comments on deferreds. I'll keep this short. These sort of
>posts don't seem to generate much/any discussion (e.g., http://bit.ly/YkSD
>which I thought was cool). I'm not sure why - I find deferreds infinitely
>interesting, and don't understand why other people don't seem to share the
>passion :-)
>
>1. If you have a deferred d1, and you call d1.chainDeferred(d2) I think it
>   makes sense for there to be a flag inside the deferred class that warns
>   you if you later call d1.addCallbacks. That's because adding more
>   functions to d1's callback/errback chain is probably not useful. (It
>   might conceivably be useful, e.g., to let you know that d1 has been
>   called). The result of d1 has been passed to d2, and because d2.callback
>   and d2.errback don't return it (see point 2 below), it is not passed
>   further down d1's chain. So it's most likely that someone adding a
>   callback to d1 after calling d1.chainDeferred doesn't really know what
>   they're doing.

Indeed.  I can't think of any case where I've wanted to add more callbacks
to a Deferred after chaining it to (from?  with?) another Deferred.

>2. Deferred.callback and Deferred.errback currently (implicitly) return
>   None.  They could be changed to return their passed argument.  That
>   would make a call to d1.chainDeferred act like a deferred fork - the
>   result would go down d2's callback chain, but also continue down the
>   callback/errback chain of d1.

But it would be catastrophic for error handling.  Suddenly all of your
errors on d1 which are handled perfectly correctly with an errback on
d2 would become unhandled failures.

>3. There's also the possibility of adding a function to the Deferred class:
>
>     def incorporateDeferred(self, d2):
>         self.callbacks.extend(d2.callbacks)
>
>   which might be what a naive reader might expect chainDeferred to do.
>   This raises the question of what you might later do with d2 (probably
>   nothing), but that's just like what happens with d1: if you call
>   d1.chainDeferred you're probably not going to return d1 to anyone. All
>   you can usefully do is fire it.

If this accounted for future changes to d2, it would make more sense.  I
think the correct direction to go in is to prevent people from trying to
add more callbacks to d1.

>I'll stop for now. There's lots more that could be said, but lets see if
>anyone's interested. I wish I were going to PyCon :-(
>
>BTW, points 1 and 2 above are key to understanding what goes on in the
>interview question code I recently posted (http://bit.ly/T3xKs).
>

Jean-Paul