[Twisted-Python] Threading and twisted

[Itamar Shtull-Trauring]

On Mon, 2006-02-06 at 13:58 -0800, David Hirschfield wrote:

> My question is: can I keep the threaded system as-is and just use the 
> xmlrpc system to add requests to the queue and pull results from it to 
> deliver to clients, as long as the xmlrpc calls maintain thread safety 
> when dealing with the shared queue and results object? Or, is it 
> generally unsafe to use non-twisted threading in conjunction with a 
> twisted app? Should I be using twisted's own thread pool and 
> runInThread() system to do the request processing work?

Twisted has a built-in thread pool. Lets say you have a function called
runTask(t) which is supposed to run on a specific task, and is known to
be thread-safe (e.g. doesn't call into Twisted). You can then do:

def xmlrpc_runtask(self, t):
    return twisted.internet.threads.deferToThread(runTask, t)

See
http://twistedmatrix.com/projects/core/documentation/howto/threading.html

You can do something similar with your own thread pool. The key is that
threads can schedule callables to run in Twisted thread using
reactor.callFromThread(). You don't want to do anything blocking (e.g.
reading from Queue) in the Twisted thread.