[Twisted-Python] Mixing generator style (yield+twisted.flow) functions with deferreds?

Marcin Kasperski Marcin.Kasperski at softax.com.pl
Tue Mar 21 06:36:45 MST 2006


I develop some twisted application using 'traditional' method 
(plenty of deferreds and small callbacks). Works well, but is 
not too readable and a bit difficult to maintain. Therefore, I 
would like to move into twisted.flow-style coding (single longer 
function returning flow.Cooperate() from time to time, wrapped 
with flow.Deferred).

There is a problem nevertheless. I still have some functions 
which return deferreds and which results are needed in the 
processing. Is it possible to somehow wrap them and keep 
yield-style of coding?

To explain what do I mean, take a look at the code below, written 
using generators. It is clean, nice and easy to maintain, but at 
the same time well-behaving in cooperative environment. Just 
what I want to use.

  from __future__ import generators
  from twisted.internet import reactor, defer
  from twisted.flow import flow

  def someFunction(name):
    return flow.Deferred( someFunctionBody(name) )

  def someFunctionBody(name):
     x = 1
     print name, x
     yield flow.Cooperate()
     x = x+1
     print name, x
     yield flow.Cooperate()
     x = x+1
     print name, x
     yield flow.Cooperate()
     x = x+1    
     print name, x
     yield x

  d = defer.DeferredList([someFunction('a'), someFunction('b')])
  d.addCallback(lambda _: reactor.stop())
  reactor.run() 

What is the problem? Imagine that somewhere within the function I 
need to call some deferred-returning function (and use its 
result in the further computation)? For instance so:

  def someFunctionBody(name):
     x = 1
     print name, x
     yield flow.Cooperate()
     x = x+1
     print name, x
     yield flow.Cooperate()
     d = functionReturningDeferredWhichFiresInteger(x)
     #### and here some magic to extract value from d to x
     #### would be so nice
     print name, x
     yield flow.Cooperate()
     x = x+1    
     print name, x
     yield x

Is it possible to apply some syntax glue to keep the code in one 
piece, without splitting the function and callback-ing 
explicite? I do *not* want to call deferred synchronously, I 
just search for opportunity to have nicely-looking code. At the 
first sight I doubt, but ... I would never believe that the 
generator-based code is possible until I saw it...

If the general solution is not possible, another question. What 
if, in the critical place, I need to call computation-intensive 
database function (currently deferred using enterprise api)?




More information about the Twisted-Python mailing list