[Twisted-Python] finer control of deferToThread ?

[Dave Peticolas]

Hey Ed, I keep meaning to check this out. What's the minimum version
of Twisted you need with it?

thanks,
dave

Ed Suominen wrote:
> You can do priority queuing to one or more threads using AsynQueue,
> http://foss.eepatents.com/AsynQueue or python-asynqueue in Debian. Use
> multiple threads and queue up the compute-intensive calls with a low
> priority and the other stuff with higher priorities.
> 
> However, you will need to chop up your compute-intensive stuff into
> smaller pieces for this to be helpful. (That's good asynchronous
> processing practice, generally.) The priority queuing is only effective
> at deciding which calls to dispatch next, and each thread call is on its
> own once it is dispatched from the queue. Each call uses an entire
> thread for its entire duration, and will keep the queue from dispatching
> anything else to that thread while it's squatting on it, no matter how
> low-priority it is.
> 
> Best regards, Ed
> 
> Josef Novak wrote:
>> Hi,
>>   I am writing a twisted application using StarPY fastagi API for
>> Asterisk.  My application involves answering user calls, and then
>> based on their responses to an IVR dialogue, running some some
>> compute-intensive applications on the same server, after they hang up.
>>   At the moment I am running the compute-intensive application (3rd
>> party code) in a deferred.deferToThread.  The application works fine,
>> however if the number of callers goes above 2-3, and one of the
>> compute-intensive applications from a previous call has not finished
>> up, the audio for the call gets very jumpy because of CPU usage.
>>   This compute-intensive process needs to be run immediately after
>> hangup, and I'd prefer to take care of everything on the same machine
>> (rather than send the compute-intensive application request somewhere
>> else).
>>
>>   What I'd like to do is pause the thread with the
>> deferred.deferToThread process any time a new  call comes in (this
>> sort of violates what I'm saying above but the number of calls going
>> to a particular ASterisk trunk is limited so in most cases this would
>> never result in more than a couple-seconds delay, which is
>> acceptable).
>>
>>   Is there any way to control these deferred.deferToThread objects in
>> a more fine-grained manner?  Say from a reactor factory?  Can I
>> register them separately somehow, and then pause these
>> compute-intensive applications temporarily every time a new call comes
>> in?  I'm imagining something as simple as as ctrl+z and $ fg linux
>> terminal commands... but I appreciate that it is probably not so
>> straightforward.
>>
>>   -Joe
>>
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>>
>>
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>