[Twisted-Python] finer control of deferToThread ?

Ed Suominen general at eepatents.com
Sat Dec 15 10:50:02 EST 2007

You can do priority queuing to one or more threads using AsynQueue,
http://foss.eepatents.com/AsynQueue or python-asynqueue in Debian. Use
multiple threads and queue up the compute-intensive calls with a low
priority and the other stuff with higher priorities.

However, you will need to chop up your compute-intensive stuff into
smaller pieces for this to be helpful. (That's good asynchronous
processing practice, generally.) The priority queuing is only effective
at deciding which calls to dispatch next, and each thread call is on its
own once it is dispatched from the queue. Each call uses an entire
thread for its entire duration, and will keep the queue from dispatching
anything else to that thread while it's squatting on it, no matter how
low-priority it is.

Best regards, Ed

Josef Novak wrote:
> Hi,
>   I am writing a twisted application using StarPY fastagi API for
> Asterisk.  My application involves answering user calls, and then
> based on their responses to an IVR dialogue, running some some
> compute-intensive applications on the same server, after they hang up.
>   At the moment I am running the compute-intensive application (3rd
> party code) in a deferred.deferToThread.  The application works fine,
> however if the number of callers goes above 2-3, and one of the
> compute-intensive applications from a previous call has not finished
> up, the audio for the call gets very jumpy because of CPU usage.
>   This compute-intensive process needs to be run immediately after
> hangup, and I'd prefer to take care of everything on the same machine
> (rather than send the compute-intensive application request somewhere
> else).
>   What I'd like to do is pause the thread with the
> deferred.deferToThread process any time a new  call comes in (this
> sort of violates what I'm saying above but the number of calls going
> to a particular ASterisk trunk is limited so in most cases this would
> never result in more than a couple-seconds delay, which is
> acceptable).
>   Is there any way to control these deferred.deferToThread objects in
> a more fine-grained manner?  Say from a reactor factory?  Can I
> register them separately somehow, and then pause these
> compute-intensive applications temporarily every time a new call comes
> in?  I'm imagining something as simple as as ctrl+z and $ fg linux
> terminal commands... but I appreciate that it is probably not so
> straightforward.
>   -Joe
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

More information about the Twisted-Python mailing list