[Twisted-Python] pb and deferToThread for geographic data services - Newbie woes

[Jeff Cavner]

I am trying to use Twisted to provide a high volume (with lots of
simultaneous requests) set of services that process and deliver several
different types of geographic data.  I am brand new to Twisted.  But a user
would come into the system through a Turbogears controller, where I intend
to write a Twisted perspective broker client to a pb server.  The server
will have different functions that ideally would operate in an asynchronous
fashion.   That is, the client would make a request for some processing on
one portion of the problem, while returning control and then another remote
function would be called to work on another part of the problem.  My
thinking was that I would have to use pb with deferToThread.  But I am
having a hard time finding any good examples that use both pb and
deferToThread.  In other words for a simple pb example like the following I
am unsure of where to add the deferToThread bits.

 

from twisted.spread import pb
from twisted.internet import reactor
from twisted.python import util
 
factory = pb.PBClientFactory()
reactor.connectTCP("localhost", 8789, factory)
d = factory.getRootObject()
d.addCallback(lambda object: object.callRemote("echo", "hello network"))
d.addCallback(lambda echo: 'server echoed: '+echo)
d.addErrback(lambda reason: 'error: '+str(reason.value))
d.addCallback(util.println)
d.addCallback(lambda _: reactor.stop())
reactor.run()

 

 

My attempt to modify the code above to take advantage of two remote function
calls (one long, and one short) to emulate longer processing times by a
function that should run while control is returned and the another function
called is woefully synchronous.

 

Here is that code:

 

from twisted.spread import pb

from twisted.internet import reactor

from twisted.python import util

from twisted.internet import threads

 

 

def getWMS():

    d.addCallback(lambda object: object.callRemote("long", "hello long"))

    d.addCallback(lambda long: 'server echoed: '+long)

    d.addErrback(lambda reason: 'error: '+str(reason.value))

    

 

def gotWMS(someobject):

    

    d.addCallback(util.println)

 

    

 

def getShort():

   e.addCallback(lambda object: object.callRemote("short", "hello short"))

   e.addCallback(lambda echo: 'server echoed: '+echo)

   e.addErrback(lambda reason: 'error: '+str(reason.value))

 

def gotShort(someobject):

 

   e.addCallback(util.println)

 

  

factory = pb.PBClientFactory()

reactor.connectTCP("localhost", 8789, factory)   

   

d = factory.getRootObject()

mythread = threads.deferToThread(getWMS)

mythread.addCallback(gotWMS)

 

e = factory.getRootObject()

mythread2 = threads.deferToThread(getShort)

mythread2.addCallback(gotShort)

 

reactor.run()

 

The long call always returns first and therefore stalls the return from the
short function. I was hoping, expecting to get the short call response back
before the long.  I am sure I am not thinking about any of this correctly.
Any pointers?????  Additionally does one have to get the rootObject for each
function call as I have done here?

 

Jeff Cavner

jcavner at edac.unm.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20090326/2c2e846d/attachment.htm