[Twisted-Python] Re: flow.Zip

Clark C. Evans cce at clarkevans.com
Wed Apr 23 10:33:29 EDT 2003


On Wed, Apr 23, 2003 at 08:57:50AM +0200, Philippe Lafoucrière wrote:
| > Cooperate() is called when an iterator would block, thus,
| > the flow module allows other tasks (connections, users)
| > to continue.   If you look at the source for flow.Threaded
| > you will see that it raises Cooperate() when the database
| > hasn't produced data yet.
| 
| Ho well. So, Cooperate() is a kind de defered (for flows) ?

Well, Cooperate is the whole reason for flow... otherwise you
could just use recursive generators.   Basically, in one of 
your generators (Threaded(QueryIterator())), your code would 
block waiting for a row, etc.  So rather than blocking, Threaded
returns Cooperate.   This return value is handled by the wrappers
all the way up the call chain.  At the top of the call chain is
Deferred, and when Deferrerd gets a Cooperate, it does a 
callLater(0, self), so that other processes can run.  


Suppose you have generator X -> Y -> Z, the picture 
then looks like...

     Wrap  <--  Z
      |
      v
     Wrap  <--> Y
      |   
      v
     Wrap  <--> X
      |
      V
     Deferred
      |
      V
     reactor

So, rather than X talking to Y, talking to Z, each of
your generators is talking to a wrapper instead, which
tries to be as transparent as possible.   The 
wrapper has the following behavior:

    0.  If a value is returned from a generator to 
        a wrapper, it is given to the previous generator
        in the chain; at the bottom of the chain it is
        added to an array which is sent to the deferred's
        callback.  Failures are handled the same way.

    1.  If the value returned is a wrapper, then it adds
        the new wrapper to the top of the stack and 
        starts to process it.

    2.  When Cooperate is returned from a generator, 
        it is sent all the way down the Wrapper linked
        list so that the reactor can re-schedule the
        processing chain (micro-thread).

So, I hope this helps.  Cooperate is the whole reason
for flow... without the need to 'pause' the whole stack
one wouldn't need the wrapper at all.  

;) Clark




More information about the Twisted-Python mailing list