[Twisted-Python] Thoughts on Deferred
Clark C. Evans
cce at clarkevans.com
Thu Mar 6 10:03:25 EST 2003
On Thu, Mar 06, 2003 at 05:34:29AM -0600, Glyph Lefkowitz wrote:
| On Thursday, March 6, 2003, at 12:02 AM, Clark C. Evans wrote:
| > 1. The entire deferred processing chain is quite nice,
| > although the bulk of the time my callbacks don't take
| > any arguments. I suspect that most people who do have
| > callback args can just make an object and pass a bound
| > method as their callback instead of the 'args' mechanism.
| Interesting idea, but I'm not sure about it yet. Need to think it over.
Ok. I'll post a patch for approval.
| > 2. The error handling could use help.
| You don't seem to understand it completely. See itamar's post.
I think I was proposing an functionally equivalent option to
the current implementation. After some thought, there really
isn't an advantage either way, although your implementation
could be ever-so-slightly-faster. Let's drop this one.
| > 3. Deferred also "artifically limits" so that the entire callback
| > tree can only be done once. This involves a hack
| > to solve the problem; but with a slightly different _runCallbacks
| > plus a __init__ flag, this need not be the case.
| If you actually try this and figure out how to get it work with the
| cases that Deferreds are used, I would be interested in seeing it,
| since this is a common criticism. However, Deferreds are not event
| broadcasters, they are deferred results, and requests that are made
| once should only be answered once.
Yes, this just doesn't fit since I want to provide for "partial"
results along the way; aka a database query or a fragment of
a file, etc.
| The following behavior, for example:
| d = Deferred()
Is this common?
I think in a multi-result case, you can't add additional
callbacks once 'callback' has been invoked; this restriction
is needed since you don't want to cache every value
just-in-case another callback would be added later.
| only calls 'foo' and 'baz' once, which makes perfect sense: each
| callback is called as soon as possible with the result at that point in
| the chain.
Right; and any modifications would maintain the current behavior. The
implementation path would add an additional flag to Deferred to indicate
if it was a multi-result thingy. I'm still trying to figure out how
to send a 'finished' message down the chain. I think this is somewhat
related to consumer/producer as well... a bit more thought is needed.
| In a "multi-"deferred case, though, what happens? We add another
| 'd.callback(boz)' to the end. Then what? does 'foo' get called once
| and 'baz' get called once? 'foo' called twice and 'baz' called once?
| 'foo' and 'baz' both called twice with foo and with bar?
You'd have one more call of 'foo' and 'bar', 'foo' passed 'boz',
assuming that the callback(bar) happened after addCallback(baz).
I can't see how you'd make it work otherwise, beacuse you don't
really want to cache the value.
| This makes callback execution highly dependent upon the order in which
| callbacks are registered.
It is already hightly dependent on this... addCallback(foo) followed
by addCallback(bar) is very different than addCallback(bar) followed
by addCallback(foo). ;)
| Having to care about this on a regular basis smacks of the same
| sort of mandatory awareness of order-of-execution that thread
| programming brings.
Agreed. So you probably could'nt support the full semantics, i.e.,
the call chain would be much less dynamic.
| > 4. Lastly, I'd like to see other "state" variables for the deferred.
| > Essentially, what I see is a process flow, aka state transition
| > mechnanism emerging. For an SQL query, one of the states
| > is 'good', 'bad', and 'finished'. Perhaps I'm a bit off here,
| > but being able to handle more than two states could be useful.
| This is why the Failure object has an exception type. See the trap
| method. For things that aren't errors you should be able to
| encapsulate that state information in the result.
Ok. I was just trying to differentate between a 'chunk' notify
and a finish message. Probably this is another difference
between the two Deferred and MultiDeferred. Perhaps they should
stay distinct then.
More information about the Twisted-Python