Ok I see this will definitely not work as we are talking about threads and not processes so os.kill is of no use to me. I guess there is no way to track these new threads and stop/pause/restart them based on some other decision logic? Would have been nice if there were though... -joe
<br><br><div><span class="gmail_quote">2007/12/16, Josef Novak <<a href="mailto:josef.robert.novak@gmail.com">josef.robert.novak@gmail.com</a>>:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>Hmm. This seems like a pretty nice solution, except that it seems it will be blocking incoming calls during processing. What I really need to do is pause the deferToThread objects based on their pid.<br><br>How can I obtain the pid of a new deferToThread object? If I can obtain it, then I can just keep a persistent dictionary containing all the running compute-intensive processes, and unregister each one in a callback after it finishes up. Then, every time a new call comes in, I check my persistent pid dictionary/list,
<br>for pid in pid_list:<br> os.kill(pid, signal.SIGSTOP)<br><br>and every time a call finishes, I check the number of current calls, and then if we're back to zero, we resume the compute intensive processes again:<br>
if parentObj.live_calls == 0::<br> for pid in parentObj.pid_list:<br> os.kill(pid, signal.SIGCONT)<br><br>so, my new question is... <br><span style="font-weight: bold;">How can I obtain the pid of a new deferToThread object?
</span> <br><br>I'm pretty sure that, while not very elegant, this should solve my problem - but please let me know if this sounds really, cockamamey!<div><span class="e" id="q_116e1071bc43ff3c_1"><br><br>-Joe<br><br>
> ------------------------------<br>> <br>
> Message: 2<br>> Date: Sat, 15 Dec 2007 07:50:02 -0800<br>> From: Ed Suominen <<a href="mailto:general@eepatents.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">general@eepatents.com
</a>><br>> Subject: Re: [Twisted-Python] finer control of deferToThread ?
<br>> To: Twisted general discussion <<a href="mailto:twisted-python@twistedmatrix.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">twisted-python@twistedmatrix.com</a>><br>> Message-ID: <
<a href="mailto:4763F7AA.2090800@eepatents.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">4763F7AA.2090800@eepatents.com
</a>><br>> Content-Type: text/plain; charset=ISO-8859-1<br>> <br>> You can do priority queuing to one or more threads using AsynQueue,<br>> <a href="http://foss.eepatents.com/AsynQueue" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
http://foss.eepatents.com/AsynQueue
</a> or python-asynqueue in Debian. Use<br>> multiple threads and queue up the compute-intensive calls with a low<br>> priority and the other stuff with higher priorities.<br>> <br>> However, you will need to chop up your compute-intensive stuff into
<br>> smaller pieces for this to be helpful. (That's good asynchronous<br>> processing practice, generally.) The priority queuing is only effective<br>> at deciding which calls to dispatch next, and each thread call is on its
<br>> own once it is dispatched from the queue. Each call uses an entire<br>> thread for its entire duration, and will keep the queue from dispatching<br>> anything else to that thread while it's squatting on it, no matter how
<br>> low-priority it is.<br>> <br>> Best regards, Ed<br>> <br>> Josef Novak wrote:<br>> > Hi,<br>> > I am writing a twisted application using StarPY fastagi API for<br>> > Asterisk. My application involves answering user calls, and then
<br>> > based on their responses to an IVR dialogue, running some some<br>> > compute-intensive applications on the same server, after they hang up.<br>> > At the moment I am running the compute-intensive application (3rd
<br>> > party code) in a deferred.deferToThread. The application works fine,<br>> > however if the number of callers goes above 2-3, and one of the<br>> > compute-intensive applications from a previous call has not finished
<br>> > up, the audio for the call gets very jumpy because of CPU usage.<br>> > This compute-intensive process needs to be run immediately after<br>> > hangup, and I'd prefer to take care of everything on the same machine
<br>> > (rather than send the compute-intensive application request somewhere<br>> > else).<br>> ><br>> > What I'd like to do is pause the thread with the<br>> > deferred.deferToThread
process any time a new call comes in (this
<br>> > sort of violates what I'm saying above but the number of calls going<br>> > to a particular ASterisk trunk is limited so in most cases this would<br>> > never result in more than a couple-seconds delay, which is
<br>> > acceptable).<br>> ><br>> > Is there any way to control these deferred.deferToThread objects in<br>> > a more fine-grained manner? Say from a reactor factory? Can I<br>> > register them separately somehow, and then pause these
<br>> > compute-intensive applications temporarily every time a new call comes<br>> > in? I'm imagining something as simple as as ctrl+z and $ fg linux<br>> > terminal commands... but I appreciate that it is probably not so
<br>> > straightforward.<br>> ><br>> > -Joe<br>> ><br>> > _______________________________________________<br>> > Twisted-Python mailing list<br>> > <a href="mailto:Twisted-Python@twistedmatrix.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
Twisted-Python@twistedmatrix.com</a><br>> > <a href="http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
</a><br>> ><br>> ><br>
> ><br>> <br>> <br>> <br>> ------------------------------<br>> <br>> _______________________________________________<br>> Twisted-Python mailing list<br>> <a href="mailto:Twisted-Python@twistedmatrix.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
Twisted-Python@twistedmatrix.com</a><br>> <a href="http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
</a><br>> <br>> <br>> End of Twisted-Python Digest, Vol 45, Issue 18
<br>> **********************************************<br>> <br>
</span></div></blockquote></div><br>