[Twisted-Python] twisted performance

Itamar Shtull-Trauring itamar at itamarst.org
Wed Dec 7 12:15:43 EST 2005


On Wed, 2005-12-07 at 02:04 -0800, Alec Matusis wrote:
> I am running Twisted 1.3 server with a fairly large number of clients.
> The hardware is two 64 bit 3.0 GHz Xeons with HT, 4GB RAM, and it's on
> 2.6.11 kernel.
> I am using poll reactor.
> Currently, when the number of clients approaches 5000, the "top" shows
> 99% CPU load for twistd process, the event loop slows down and weird
> racing conditions show up. 

Could you try this experiment with Twisted 2.1, ideally with the latest
version of Python? There were a number of algorithmic improvements since
1.3 (the one that comes to mind in this case is the scheduler.)

One way to discover if the OS-level poll() is the problem is to use
oprofile; you should be able to use it to see how much time is spent in
there. Before that however, you'd want to use the Python profiler to
figure out if there's any obvious hotspots.






More information about the Twisted-Python mailing list