[Twisted-Python] Re: flow.Zip
Clark C. Evans
cce at clarkevans.com
Wed Apr 23 10:33:29 EDT 2003
On Wed, Apr 23, 2003 at 08:57:50AM +0200, Philippe Lafoucrière wrote:
| > Cooperate() is called when an iterator would block, thus,
| > the flow module allows other tasks (connections, users)
| > to continue. If you look at the source for flow.Threaded
| > you will see that it raises Cooperate() when the database
| > hasn't produced data yet.
| Ho well. So, Cooperate() is a kind de defered (for flows) ?
Well, Cooperate is the whole reason for flow... otherwise you
could just use recursive generators. Basically, in one of
your generators (Threaded(QueryIterator())), your code would
block waiting for a row, etc. So rather than blocking, Threaded
returns Cooperate. This return value is handled by the wrappers
all the way up the call chain. At the top of the call chain is
Deferred, and when Deferrerd gets a Cooperate, it does a
callLater(0, self), so that other processes can run.
Suppose you have generator X -> Y -> Z, the picture
then looks like...
Wrap <-- Z
Wrap <--> Y
Wrap <--> X
So, rather than X talking to Y, talking to Z, each of
your generators is talking to a wrapper instead, which
tries to be as transparent as possible. The
wrapper has the following behavior:
0. If a value is returned from a generator to
a wrapper, it is given to the previous generator
in the chain; at the bottom of the chain it is
added to an array which is sent to the deferred's
callback. Failures are handled the same way.
1. If the value returned is a wrapper, then it adds
the new wrapper to the top of the stack and
starts to process it.
2. When Cooperate is returned from a generator,
it is sent all the way down the Wrapper linked
list so that the reactor can re-schedule the
processing chain (micro-thread).
So, I hope this helps. Cooperate is the whole reason
for flow... without the need to 'pause' the whole stack
one wouldn't need the wrapper at all.
More information about the Twisted-Python