<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<style>
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
pre
{margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Arial;
color:windowtext;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>I am trying to use Twisted to provide a high volume (with lots
of simultaneous requests) set of services that process and deliver several
different types of geographic data. I am brand new to Twisted. But a
user would come into the system through a Turbogears controller, where I intend
to write a Twisted perspective broker client to a pb server. The server
will have different functions that ideally would operate in an asynchronous
fashion. That is, the client would make a request for some
processing on one portion of the problem, while returning control and then
another remote function would be called to work on another part of the problem.
My thinking was that I would have to use pb with deferToThread. But I am
having a hard time finding any good examples that use both pb and
deferToThread. In other words for a simple pb example like the following
I am unsure of where to add the deferToThread bits.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<pre><span class=py-src-keyword><font size=2 face="Courier New"><span
style='font-size:10.0pt'>from</span></font></span> <span class=py-src-variable>twisted</span>.<span
class=py-src-variable>spread</span> <span class=py-src-keyword>import</span> <span
class=py-src-variable>pb</span><o:p></o:p></pre><pre><span
class=py-src-keyword><font size=2 face="Courier New"><span style='font-size:
10.0pt'>from</span></font></span> <span class=py-src-variable>twisted</span>.<span
class=py-src-variable>internet</span> <span class=py-src-keyword>import</span> <span
class=py-src-variable>reactor</span><o:p></o:p></pre><pre><span
class=py-src-keyword><font size=2 face="Courier New"><span style='font-size:
10.0pt'>from</span></font></span> <span class=py-src-variable>twisted</span>.<span
class=py-src-variable>python</span> <span class=py-src-keyword>import</span> <span
class=py-src-variable>util</span><o:p></o:p></pre><pre><font size=2
face="Courier New"><span style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre><span
class=py-src-variable><font size=2 face="Courier New"><span style='font-size:
10.0pt'>factory</span></font></span> = <span class=py-src-variable>pb</span>.<span
class=py-src-variable>PBClientFactory</span>()<o:p></o:p></pre><pre><span
class=py-src-variable><font size=2 face="Courier New"><span style='font-size:
10.0pt'>reactor</span></font></span>.<span class=py-src-variable>connectTCP</span>(<span
class=py-src-string>"localhost"</span>, <span class=py-src-number>8789</span>, <span
class=py-src-variable>factory</span>)<o:p></o:p></pre><pre><span
class=py-src-variable><font size=2 face="Courier New"><span style='font-size:
10.0pt'>d</span></font></span> = <span class=py-src-variable>factory</span>.<span
class=py-src-variable>getRootObject</span>()<o:p></o:p></pre><pre><span
class=py-src-variable><font size=2 face="Courier New"><span style='font-size:
10.0pt'>d</span></font></span>.<span class=py-src-variable>addCallback</span>(<span
class=py-src-keyword>lambda</span> <span class=py-src-variable>object</span>: <span
class=py-src-variable>object</span>.<span class=py-src-variable>callRemote</span>(<span
class=py-src-string>"echo"</span>, <span class=py-src-string>"hello network"</span>))<o:p></o:p></pre><pre><span
class=py-src-variable><font size=2 face="Courier New"><span style='font-size:
10.0pt'>d</span></font></span>.<span class=py-src-variable>addCallback</span>(<span
class=py-src-keyword>lambda</span> <span class=py-src-variable>echo</span>: <span
class=py-src-string>'server echoed: '</span>+<span class=py-src-variable>echo</span>)<o:p></o:p></pre><pre><span
class=py-src-variable><font size=2 face="Courier New"><span style='font-size:
10.0pt'>d</span></font></span>.<span class=py-src-variable>addErrback</span>(<span
class=py-src-keyword>lambda</span> <span class=py-src-variable>reason</span>: <span
class=py-src-string>'error: '</span>+<span class=py-src-variable>str</span>(<span
class=py-src-variable>reason</span>.<span class=py-src-variable>value</span>))<o:p></o:p></pre><pre><span
class=py-src-variable><font size=2 face="Courier New"><span style='font-size:
10.0pt'>d</span></font></span>.<span class=py-src-variable>addCallback</span>(<span
class=py-src-variable>util</span>.<span class=py-src-variable>println</span>)<o:p></o:p></pre><pre><span
class=py-src-variable><font size=2 face="Courier New"><span style='font-size:
10.0pt'>d</span></font></span>.<span class=py-src-variable>addCallback</span>(<span
class=py-src-keyword>lambda</span> <span class=py-src-variable>_</span>: <span
class=py-src-variable>reactor</span>.<span class=py-src-variable>stop</span>())<o:p></o:p></pre><pre><span
class=py-src-variable><font size=2 face="Courier New"><span style='font-size:
10.0pt'>reactor</span></font></span>.<span class=py-src-variable>run</span>()<o:p></o:p></pre>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>My attempt to modify the code above to take advantage of two
remote function calls (one long, and one short) to emulate longer processing
times by a function that should run while control is returned and the another
function called is woefully synchronous.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Here is that code:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>from twisted.spread import pb<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>from twisted.internet import reactor<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>from twisted.python import util<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>from twisted.internet import threads<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>def getWMS():<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> d.addCallback(lambda object:
object.callRemote("long", "hello long"))<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> d.addCallback(lambda long: 'server
echoed: '+long)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> d.addErrback(lambda reason: 'error:
'+str(reason.value))<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>def gotWMS(someobject):<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> d.addCallback(util.println)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>def getShort():<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> e.addCallback(lambda object:
object.callRemote("short", "hello short"))<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> e.addCallback(lambda echo: 'server echoed:
'+echo)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> e.addErrback(lambda reason: 'error:
'+str(reason.value))<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>def gotShort(someobject):<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> e.addCallback(util.println)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>factory = pb.PBClientFactory()<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>reactor.connectTCP("localhost", 8789,
factory) <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>d = factory.getRootObject()<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>mythread = threads.deferToThread(getWMS)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>mythread.addCallback(gotWMS)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>e = factory.getRootObject()<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>mythread2 = threads.deferToThread(getShort)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>mythread2.addCallback(gotShort)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>reactor.run()<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>The long call always returns first and therefore stalls the
return from the short function. I was hoping, expecting to get the short call
response back before the long. I am sure I am not thinking about any of
this correctly. Any pointers????? Additionally does one have to get
the rootObject for each function call as I have done here?<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Jeff Cavner<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>jcavner@edac.unm.edu<o:p></o:p></span></font></p>
</div>
</body>
</html>