[Twisted-Python] Soon to be not-a-newbie?

Maarten ter Huurne maarten at treewalker.org
Sat Jan 26 07:41:55 EST 2008


On Saturday 26 January 2008, Tristan Seligmann wrote:
> * Maarten ter Huurne <maarten at treewalker.org> [2008-01-26 12:51:27 +0100]:
> > The low-level flow of code (control going back to the reactor) is
> > obscured, but I think that is an advantage rather than a disadvantage,
> > since the high-level flow of code (the sequence of operations) becomes
> > more clear. The flow of data is explicit in both cases: it is either
> > passed as callback parameters or as return values from "yield".
>
> In the vast majority of network applications, I believe deferred
> operations are not sequenced in a linear fashion; they're happening in
> an overlapping and concurrent fashion. The problem with data flow in the
> case of "yield" is that every time "yield" is invoked, all sorts of
> state can get mutated around you that wouldn't normally happen when
> calling another function. For a normal function, this also happens when
> you return control to your caller, but by that time you don't care; but
> in a generator, you're going to be resumed again. The "yield" keyword
> doesn't seem to act as a very good "red flag" for this kind of context
> switch, especially once it starts getting buried in a more complex
> expression.

The outside world changing is only a problem if you gather information about 
the outside world in stages, since you could end up with inconsistent data. 
However, this problem exists whether you store the gathered data in the 
local variables of an inline callback or in a data structure passed to 
non-inlined callbacks.

Whether "yield" is a worse red flag than separate functions I cannot really 
tell, since I wrote only one big application with Twisted and there I do 
all data gathering in a single operation. That is likely to change in the 
future though, when it will start fetching information from other servers.

> > There are quite a few cases in which the dependencies between the
> > operations force sequential processing. In those cases, inline
> > callbacks are useful.
>
> I've personally encountered very few cases where there are more than a
> handful of operations in sequence.

True, but even for a sequence of 3 operations I think it's already 
worthwhile.

> > For example, to serve a web page, I want to authenticate the user, then
> > run a database query and finally present the result.
>
> Generally in my code, these are being handled by three different layers
> of code that are mostly hooked up by Twisted and other frameworks; for
> example, authentication is being handled by cred/guard in my web
> applications. Thus, there isn't really any single function where all
> these operations are plumbed together.

I do authenticate using cred. I don't use Nevow though: I played with it a 
bit and decided it was too complex for my taste (I was always wondering 
exactly when a substitution would be performed). So I wrote something 
specifically for our application: because it's less flexible it's also 
easier to use.

> > If the code using inline callbacks looks like this, there is no problem
> > in testing the parts separately:
> >
> > 	result1 = yield function1(arg)
> > 	result2 = yield function2(result1)
>
> But if it really looks like that, there's not much point in writing that
> instead of this:
>
>     result1 = function1(arg)
>     result2 = result1.addCallback(function2)

It doesn't literally look like that. What I mean is that if there is enough 
functionality to make it worth dedicating test cases to, then it's probably 
a good idea to put that functionality in a separate function. This is true 
for both inline and non-inline callbacks, and even for synchronous code.

The cases where inline callbacks really help are when the callback functions 
contain only 1 or 2 lines, for example logging a failure and propagating it 
using a different exception type.

Bye,
		Maarten
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part.
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20080126/a5419011/attachment.pgp 


More information about the Twisted-Python mailing list