[Twisted-Python] Soon to be not-a-newbie?

Tristan Seligmann mithrandi at mithrandi.za.net
Sat Jan 26 07:01:07 EST 2008

* Maarten ter Huurne <maarten at treewalker.org> [2008-01-26 12:51:27 +0100]:

> On Saturday 26 January 2008, Tristan Seligmann wrote:
> > * Nitro <nitro at dr-code.org> [2008-01-25 23:42:41 +0100]:
> > >>> the recent spate of inlineCallbacks-related posts alarms me.
> > >
> > > Why? Using inlineCallbacks just helps writing convenient syntax
> > > utilizing deferreds. If you ever wrote code which uses lots of
> > > deferreds, you know that the code is (visually) lacking a decent
> > > structure. It's just not the way you think and makes things hard to
> > > read. inlineCallbacks just turns this into a nice, linear piece of code
> > > again (linear not to be confused with synchronous!).
> >
> > The problem I have with this is that I find that it actually obscures
> > the flow of the code and data.
> The low-level flow of code (control going back to the reactor) is obscured, 
> but I think that is an advantage rather than a disadvantage, since the 
> high-level flow of code (the sequence of operations) becomes more clear. 
> The flow of data is explicit in both cases: it is either passed as callback 
> parameters or as return values from "yield".

In the vast majority of network applications, I believe deferred
operations are not sequenced in a linear fashion; they're happening in
an overlapping and concurrent fashion. The problem with data flow in the
case of "yield" is that every time "yield" is invoked, all sorts of
state can get mutated around you that wouldn't normally happen when
calling another function. For a normal function, this also happens when
you return control to your caller, but by that time you don't care; but
in a generator, you're going to be resumed again. The "yield" keyword
doesn't seem to act as a very good "red flag" for this kind of context
switch, especially once it starts getting buried in a more complex

> > Most deferred operations have no need to 
> > be serialized, and shouldn't be; the linear nature of
> > inlineCallbacks-style makes it very easy to accidentally serialize
> > operations that could otherwise run in parallel instead. Even if you
> > write the generator correctly, it's not as obvious what the actual flow
> > is since it has been crunched into a linear-looking function.
> There are quite a few cases in which the dependencies between the operations 
> force sequential processing. In those cases, inline callbacks are useful.

I've personally encountered very few cases where there are more than a
handful of operations in sequence.

> For example, to serve a web page, I want to authenticate the user, then run 
> a database query and finally present the result.

Generally in my code, these are being handled by three different layers
of code that are mostly hooked up by Twisted and other frameworks; for
example, authentication is being handled by cred/guard in my web
applications. Thus, there isn't really any single function where all
these operations are plumbed together.

> > Finally, it is extremely hard to unit test a generator using
> > inlineCallbacks, as there is no easy way of resuming the generator
> > at certain points with certain state to test each part of the
> > generator.
> If the code using inline callbacks looks like this, there is no problem in 
> testing the parts separately:
> 	result1 = yield function1(arg)
> 	result2 = yield function2(result1)

But if it really looks like that, there's not much point in writing that
instead of this:

    result1 = function1(arg)
    result2 = result1.addCallback(function2)

mithrandi, i Ainil en-Balandor, a faer Ambar
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20080126/0f45c448/attachment.pgp 

More information about the Twisted-Python mailing list