Hey guys,<br><br>Thanks for your responses.<br>Nathan your pointer to the "deferToThread" really helped.<br><br>I finally did this:<br><br>ran my external script in a thread using: <br>threads.deferToThread(test.main....)<br>
<br>and started a function called status_check in another thread<br>threads.deferToThread(self.check_status)<br><br>then immidiately returned control to reactor.<br><br>def check_status:<br> self.testprocess_status = self.parent_conn.recv()<br>
<br>now, this call on pipe blocks the thread not the reactor main loop.<br>in the reactor when the value is changed of this self.testprocess_status I ping my client with the new status.<br><br>"Threads are twisted :)"<br>
<br>Thanks<br>Naman <br><br><div class="gmail_quote">On Tue, Oct 27, 2009 at 4:31 PM, Valeriy Pogrebitskiy <span dir="ltr"><<a href="mailto:vpogrebi@verizon.net">vpogrebi@verizon.net</a>></span> wrote:<br>
<div> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div link="blue" vlink="blue" lang="EN-US"><div><p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Naman,</span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Another problem you might (possibly) be
encountering – is the way your server and/or client protocol is implemented.
If you just inherit from “plain” twisted.internet.protocol.Protocol
– you are automatically using write () and dataReceived() methods that do
not implement buffering (it is left up to developer). To enforce that line
(string) oriented communication works correctly – you might want to refactor
to use twisted.protocols.basic.LineReceiver instead. </span></font></p><div class="im">
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<div>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Kind regards,</span></font></p>
<p class="MsoNormal"><font color="navy" face="Times New Roman" size="3"><span style="font-size: 12pt; color: navy;"> </span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Valeriy Pogrebitskiy</span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Email: <a href="mailto:vpogrebi@iname.com" target="_blank">vpogrebi@iname.com</a></span></font></p>
<p class="MsoNormal"><font color="navy" face="Times New Roman" size="3"><span style="font-size: 12pt; color: navy;"> </span></font></p>
<p class="MsoNormal"><font color="navy" face="Times New Roman" size="3"><span style="font-size: 12pt; color: navy;"> </span></font></p>
</div>
</div><p class="MsoNormal" style="margin-left: 0.5in;"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"><div class="im">-----Original Message-----<br>
<b><span style="font-weight: bold;">From:</span></b>
<a href="mailto:twisted-python-bounces@twistedmatrix.com" target="_blank">twisted-python-bounces@twistedmatrix.com</a>
[mailto:<a href="mailto:twisted-python-bounces@twistedmatrix.com" target="_blank">twisted-python-bounces@twistedmatrix.com</a>] <b><span style="font-weight: bold;">On Behalf Of </span></b>naman jain<br>
</div><div><div></div><div class="h5"><b><span style="font-weight: bold;">Sent:</span></b> Tuesday, October 27, 2009
6:36 AM<br>
<b><span style="font-weight: bold;">To:</span></b> Twisted general discussion<br>
<b><span style="font-weight: bold;">Subject:</span></b> Re: [Twisted-Python]
Regarding Twisted Matrix</div></div></span></font></p><div><div></div><div class="h5">
<p class="MsoNormal" style="margin-left: 0.5in;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<p class="MsoNormal" style="margin-right: 0in; margin-bottom: 12pt; margin-left: 0.5in;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">I
tried to implement this:<br>
<br>
parent_conn, child_conn = Pipe()<br>
<br>
f = defer.Deferred()<br>
f = threads.deferToThread(start_test.main_func, SCRIPT_PATH, TEMP_OUTPUT_PATH,
self.output_name, child_conn)<br>
<br>
response = parent_conn.recv()<br>
print response //prints like: initialization done<br>
self.transport.write(response)<br>
<br>
response = parent_conn.recv()<br>
print response // configuration done<br>
self.transport.write(response)<br>
<br>
But get the same output ie all status messages concatenated and sent at once.!<br>
<br>
[Do we need to put some deffered/callback mechanism on parent_conn.recv()
??<br>
as that is the call blocking the thread waiting for reading.]<br>
<br>
Any ideas ? How is such stuff done in twisted servers??<br>
<br>
Naman<br>
<br>
</span></font></p>
<div>
<p class="MsoNormal" style="margin-left: 0.5in;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">On Tue, Oct 27, 2009 at 7:03 AM, Nathan <<a href="mailto:nathan.stocks@gmail.com" target="_blank">nathan.stocks@gmail.com</a>> wrote:</span></font></p>
<div>
<div>
<p class="MsoNormal" style="margin-right: 0in; margin-bottom: 12pt; margin-left: 0.5in;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">On
Mon, Oct 26, 2009 at 11:44 PM, naman jain <<a href="mailto:namanvit@gmail.com" target="_blank">namanvit@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> I have a client server model in twisted, where the server spawns a thread
(<br>
> basically a test script in python that runs for about 20 mins)<br>
> I want to track the progress of the thread, and send the progress to the<br>
> client back<br>
><br>
> So, I write something like this in my server:<br>
><br>
> parent_conn, child_conn = Pipe()<br>
> thread = Process(target = start_test.main_func, args=(SCRIPT_PATH,<br>
> TEMP_OUTPUT_PATH, self.output_name, child_conn))<br>
> thread.start()<br>
><br>
> response = parent_conn.recv()<br>
> print response //prints like: initialization done<br>
> self.transport.write(response)<br>
><br>
><br>
> response = parent_conn.recv()<br>
> print response // configuration done<br>
> self.transport.write(response)<br>
><br>
> .<br>
> .<br>
> .<br>
><br>
> thread.join()<br>
><br>
><br>
> But the transport.write calls don't send at the same time. instead they
wait<br>
> for the thread to finish (coz of thread.join) and then append all the<br>
> response and send it back; like "initialization doneconfiguration<br>
> done...done"<br>
> thereby defeating the purpose of creating a thread.<br>
><br>
> How do I give the control to the reactor to write the data back, and still<br>
> keep the thread running?<br>
> or is there any other way these kinda of progress can be tracked ?<br>
> I am kinda stuck with this :(</span></font></p>
</div>
</div>
<p class="MsoNormal" style="margin-left: 0.5in;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">I'm no expert with threads, but I don't see any
deferToThread in<br>
there, which is what people on this list tend to mention whenever<br>
threading comes up. Perhaps you need to switch to the "twisted
way"<br>
to create your thread?<br>
<br>
<a href="http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.threads.html" target="_blank">http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.threads.html</a><br>
<font color="#888888"><span style="color: rgb(136, 136, 136);"><br>
~ Nathan</span></font></span></font></p>
<div>
<div>
<p class="MsoNormal" style="margin-left: 0.5in;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"><br>
_______________________________________________<br>
Twisted-Python mailing list<br>
<a href="mailto:Twisted-Python@twistedmatrix.com" target="_blank">Twisted-Python@twistedmatrix.com</a><br>
<a href="http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python" target="_blank">http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python</a></span></font></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-left: 0.5in;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
</div></div></div>
</div>
<br>_______________________________________________<br>
Twisted-Python mailing list<br>
<a href="mailto:Twisted-Python@twistedmatrix.com">Twisted-Python@twistedmatrix.com</a><br>
<a href="http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python" target="_blank">http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python</a><br>
<br></blockquote></div><br>