[Twisted-Python] twisted performance and reactor choice

[Jean-Paul Calderone]

On Tue, 14 Nov 2006 09:14:39 +0000 (GMT), Stéphane Brault <stephane_brault at yahoo.fr> wrote:
>Hi,
>
>My application has lots of concurrent connections, both in and out, since 
>it communicates with outsides sites. It seems that it is sometimes slowed 
>down, I guess because there are to many simultaneous deffered. I'd like to 
>know if I could switch reactor (my application is accessed by XML-RPC and 
>communicates with external applications through XML-RPC, Http requests and 
>access ftp sites). I also need to know how large can be the threadpool in 
>order to increase the number of simultaneous deferred.

You've gone a bit too fast here.  You guessed at the cause of the slowdown.
Instead, you need to profile the application and find out what the cause
really is.

Your question about the size of the threadpool is a bit of a non sequitur.
The threadpool, let alone its size, is unrelated to how many Deferreds might
exist in your application at once.  Unless you are explicitly using the
threadpool (or doing a significant number of DNS lookups without using
Twisted Names), the threadpool isn't even related to how many concurrent
tasks might be in progress in your application.  This is because Twisted only
uses the threadpool for an extremely limited number of purposes.  Basic
network traffic isn't one of these purposes.

Jean-Paul