[Twisted-Python] Thoughts on Deferred

[Glyph Lefkowitz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Thursday, March 6, 2003, at 12:02 AM, Clark C. Evans wrote:

>   1.  The entire deferred processing chain is quite nice,
>       although the bulk of the time my callbacks don't take
>       any arguments.  I suspect that most people who do have
>       callback args can just make an object and pass a bound
>       method as their callback instead of the 'args' mechanism.

Interesting idea, but I'm not sure about it yet.  Need to think it over.

>   2.  The error handling could use help.

You don't seem to understand it completely.  See itamar's post.

>   3.  Deferred also "artifically limits" so that the entire callback
>       tree can only be done once.   This involves a hack 
> "MultiDeferred"
>       to solve the problem; but with a slightly different _runCallbacks
>       plus a __init__ flag, this need not be the case.

If you actually try this and figure out how to get it work with the 
cases that Deferreds are used, I would be interested in seeing it, 
since this is a common criticism.  However, Deferreds are not event 
broadcasters, they are deferred results, and requests that are made 
once should only be answered once.  The following behavior, for example:

	d = Deferred()
	d.addCallback(foo)
	d.callback(bar)
	d.addCallback(baz)

only calls 'foo' and 'baz' once, which makes perfect sense: each 
callback is called as soon as possible with the result at that point in 
the chain.

In a "multi-"deferred case, though, what happens?  We add another 
'd.callback(boz)' to the end.  Then what?  does 'foo' get called once 
and 'baz' get called once?  'foo' called twice and 'baz' called once?  
'foo' and 'baz' both called twice with foo and with bar?

This makes callback execution highly dependent upon the order in which 
callbacks are registered.  Some other, similar callback abstraction 
mechanism might be useful here, but the semantics of deferreds would 
become pretty badly polluted to add something which _does_ depend on 
timing of the '.callback' method, where Deferreds have spent so much 
energy trying to get away from caring about that order.  Having to care 
about this on a regular basis smacks of the same sort of mandatory 
awareness of order-of-execution that thread programming brings.

>   4.  Lastly, I'd like to see other "state" variables for the deferred.
>       Essentially, what I see is a process flow, aka state transition
>       mechnanism emerging.   For an SQL query, one of the states
>       is 'good', 'bad', and 'finished'.  Perhaps I'm a bit off here,
>       but being able to handle more than two states could be useful.

This is why the Failure object has an exception type.  See the trap 
method.  For things that aren't errors you should be able to 
encapsulate that state information in the result.

More than two states smells like overengineering to me.  Does anyone 
else need this?

> Just some thoughts.  If you like, I could refactor the Deferred
> object as above (it'd be less code & probably cleaner), as well
> as updating the documentation and providing examples which run
> without modification.

Regardless of my criticisms I'd like to see you mail a patch to the 
list, so that I can comment on it more specifically.  Especially the 
first point: some more examples of how that would work would be helpful.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (Darwin)

iD8DBQE+ZzJJvVGR4uSOE2wRAvg8AKCEUUK4FvoOoTQzm3a2dUAVfLZCfACePMRW
ENuwW/CTSXdtI3H/GeHt/cY=
=O01d
-----END PGP SIGNATURE-----