[Twisted-Python] Soon to be not-a-newbie?

[Maarten ter Huurne]

On Saturday 26 January 2008, Tristan Seligmann wrote:
> * Nitro <nitro at dr-code.org> [2008-01-25 23:42:41 +0100]:
> >>> the recent spate of inlineCallbacks-related posts alarms me.
> >
> > Why? Using inlineCallbacks just helps writing convenient syntax
> > utilizing deferreds. If you ever wrote code which uses lots of
> > deferreds, you know that the code is (visually) lacking a decent
> > structure. It's just not the way you think and makes things hard to
> > read. inlineCallbacks just turns this into a nice, linear piece of code
> > again (linear not to be confused with synchronous!).
>
> The problem I have with this is that I find that it actually obscures
> the flow of the code and data.

The low-level flow of code (control going back to the reactor) is obscured, 
but I think that is an advantage rather than a disadvantage, since the 
high-level flow of code (the sequence of operations) becomes more clear. 
The flow of data is explicit in both cases: it is either passed as callback 
parameters or as return values from "yield".

> Most deferred operations have no need to 
> be serialized, and shouldn't be; the linear nature of
> inlineCallbacks-style makes it very easy to accidentally serialize
> operations that could otherwise run in parallel instead. Even if you
> write the generator correctly, it's not as obvious what the actual flow
> is since it has been crunched into a linear-looking function.

There are quite a few cases in which the dependencies between the operations 
force sequential processing. In those cases, inline callbacks are useful.

For example, to serve a web page, I want to authenticate the user, then run 
a database query and finally present the result. Running the query before 
authenticating the user is not something I'd recommend. In some cases it 
might be possible to start presenting results before all queries are 
finished, but in many cases that is not worth the complexity.

Writing all your routines as inline callbacks without thinking about the 
dependencies is a bad idea though.

> Finally, 
> it is extremely hard to unit test a generator using inlineCallbacks, as
> there is no easy way of resuming the generator at certain points with
> certain state to test each part of the generator.

If the code using inline callbacks looks like this, there is no problem in 
testing the parts separately:

	result1 = yield function1(arg)
	result2 = yield function2(result1)

> I'm not necessarily convinced that inlineCallbacks is always bad, but it
> certainly leads to subtle traps in most cases, while providing little
> real benefit (despite the perceived benefit).

I've converted some routines I wrote before Twisted 2.5 to inline callbacks 
and it became a lot easier for me to read. Like any tool, it can be used or 
abused, but I definately think it has its uses.

Bye,
		Maarten
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part.
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20080126/9eec3050/attachment.pgp