[Twisted-Python] Best strategies for pb Referenceables running long methods from callRemote
exarkun at twistedmatrix.com
exarkun at twistedmatrix.com
Fri Mar 18 21:40:38 EDT 2011
On 10 Mar, 11:08 pm, charlessolar at gmail.com wrote:
>I am using PB to run remote methods in a testing system at my company.
>The
>code works very well but breaks down when I start running multiple
>tests at
>once. I have tracked this down to overflowing the thread pool on the
>remote
>machines. I am wondering if anyone might have better suggestions for
>running long methods from a remote method.
>
>I coded up a sample of what I am seeing here:
>http://pastebin.com/rBPp20Ms
>
>Basically I have 1 server that calls remote_execute on many clients on
>a
>remote server. This remote_execute method starts a new method using
>threads.deferToThread and returns the defer to make the server's
>callRemote
>defer wait until the remote long method end.
>What I do in those methods is run test code that waits, blocks, sleeps,
>and
>all sorts of nasty things that make the thread take a while. In the
>example
>code I simply sleep for 20 seconds.
>
>The problem I see with this code specifically is that I run out of
>threads
>on the pool and even though I wanted all execute methods to run at the
>same
>time, I see 10 run, then 10 more, then 10 more.. etc. The testing
>depends
>on all these methods being run at the same time as they run mechanisms
>that
>depend on each other and need everyone running. When I overflow the
>thread
>pool some methods do not run until other methods stop, which makes the
>whole
>test fail.
This is how the thread pool works. It has a maximum size, which limits
the number of threads it will create to process work given to it.
Beyond that number of concurrent tasks, things will begin to get queued
up and wait for a free thread to execute them.
Each task you give to the thread pool exclusively uses one of its
threads for the entire duration of the task, regardless of what the task
consists of.
>I am not holding the GIL or blocking the reactor, which was the first
>thing
>I checked.
>
>Setting reactor.suggestThreadPoolSize(50) does help, but I do not think
>its
>the best solution, and does not work very well on our slow and older
>machines.
Using more threads is the only solution to the problem of not using
enough threads. Alternatively, look for wards to process tasks without
using threads.
Jean-Paul
More information about the Twisted-Python
mailing list