[Twisted-Python] ANN: deferred howto/tutorial

[Andrew Bennetts]

On Sat, Oct 23, 2004 at 10:40:40PM +0200, stefan wrote:
[...]
> What I also would like to see is some coding practices of how to style 
> deferred code. I got used to inner functions in class methods, which 
> for me looks clearest, but I'm sure there are other practices. A short 
> discussion on that in the docs would be great. I've attached an example 
> of my style, maybe someone with a different style can rewrite the 
> example and comment on pros and cons?
> ---------------------------------------------------------------
> class X:
>     ...
>     def y(self):
>         # do some method initialisation
>         ...
>         # callbacks
>         def oneBack(result):
[...]
>         return self.somethingDeferred().addCallback(oneBack)
> ---------------------------------------------------------------

There's a cryptic note here that partly explains the usual style within
Twisted itself:
    http://twistedmatrix.com/documents/howto/policy/coding-standard#auto13

Twisted style is usually:

class X:
    def y(self):
        ...
        return self.somethingDeferred().addCallback(self._cbOneBack)

    def _cbOneBack(self, result):
        ...

I believe the main reason for this is flexibility.  Code in Twisted tends to
be framework code or library code, rather than an actual application, so it
is written with re-use in mind.  You can't override a callback that's
embedded within a method without overriding the whole method; making the
callback a method on the class solves that.  The "_cb" or "_eb" prefix
signals that it's intended to be used as the callback or errback for
something, and also that it's not an ordinary method that you would call
directly.

It also has other secondary advantages, like making setting breakpoints in
pdb easier, although I can't say I've taken advantage of that very often (I
typically set breakpoints by inserting "import pdb; pdb.set_trace()"), but
I imagine some other developers have.

A more important advantage (if you are strict on testing) is that you can
unit test the behaviour of the callback more easily if you can call it
directly.

If nothing else, I like my functions to be as short as possible, and
embedding a large callback within a method means I now have two large
functions (the embedded function, and the method that contains it), rather
than just one.

The obvious disadvantage compared to your style is the loss of the direct
visual association of which callbacks relate to which deferreds.  I'd be
moderately interested in hearing how other people cope with that, but I
haven't really found it to be a problem.  I expect that if I did, I'd just
put comments like "# callback for deferred from self.frobnicate" at the top
of each callback and errback.  Large, hard-to-navigate classes with lots of
methods can be a sign of a suboptimal design anyway.

-Andrew.