[Twisted-web] Flamebait

[Stephen Thorne]

On Thu, Apr 15, 2004 at 01:54:25PM -0400, Jonathan Vanasco wrote:
> I couldn't think of a better title for this question...
> 
> Does anyone have any benchmarking stats on Twisted/TwistedWeb/Nevow in 
> comparison to Apache's mod_perl, mod_python or (gasp) php for general 
> tasks?
> 
> You know, basic stuff like serving static pages, generating dynamic 
> pages out of thin air, and generating dynamic pages off of parsed 
> templates.
> 
> I haven't seen comparison figures anywhere.  I assume twisted stacks up 
> impressively -- when I noticed template files being read each page 
> generation, I started to think about efficiency, disk access and 
> caching -- and hoped there were some numbers that could educate me.

I'm going to give you straight answers, because I really don't believe
in beating around the bush. I could give you curvy twisty statements
that made you feel better about using twisted instead of apache while
avoiding all figures, or I could lie to you with statistics by picking
only the ones that are favourable.

In short, twisted is slower than apache. This is a truth that you (and
we) should acknowledge. Apache is a very mature web server that has
stood the test of time. Its not the fastest and its not the best, but
its pretty darn good.

mod_php and mod_perl are blindingly fast. I can run simple mod_php
scripts faster than i can serve static files with twisted.

Twisted AND apache are still faster than the average connection. Serving
static files on a modest 800mhz machine gives me apache being twice as
fast as twisted, but twisted was able to pump through 12meg/second. If
you have a connection that exceeds 12meg/second transfers upstream, can
you please contact me regarding getting an account.

Serving woven was incredibly slow, so much so that with -n 1000 -c 100
in ab (apachebench) I actually got a timeout. (I was serving a
DirectoryListing).

So, in face of all this? Should we use twisted?

DEFINATELY. Twisted seriously cuts down on required development time for
achieving results with certain types of applications, especially
applications that trancend the web model. Anything stateful, anything
that wants to integrate with other protocols, anything that has long
running processing (defer and pickup the result several page views
later, YES PLEASE!).

Also, a while back i did develop a caching mechanism for woven pages
that allowed me to increase pageload speed by about 10fold. I think I've
still got the code laying around somewhere, it was rather simple and
very effective - based on the existing concept of woven models notifying
objects of when they change you could invalidate the cache and
regenerate on the next pageload.

Regards,
Stephen Thorne.