[Twisted-Python] Degrading under load

[Paul G]

----- Original Message ----- 
From: "Yitzchak Gale" <gale at sefer.org>
To: <twisted-python at twistedmatrix.com>
Sent: Wednesday, March 08, 2006 5:46 AM
Subject: [Twisted-Python] Degrading under load

 --- snip ---

premature optimization is just as bad as premature 
$rhymes_with_optimization. implement all of this in a single process, 
benchmark under various loads and see if you need to do anything at all. if 
you do, consider splitting the full handling of a request into separate, 
cooperative pieces with a deferred/callLater (connection handling and xml 
processing would be the natural division here). if this is insufficient (eg 
if the xml processing is sufficiently expensive), perform it in a 
threadpool. i would only split it off into a completely separate process if 
you bottleneck on the GIL, which i wouldn't consider likely in your case.

-p