jvanasco at mastersofbranding.com
Thu Apr 15 23:05:06 MDT 2004
Thanks for the input!
I've been working on this one project in my spare time for 3 years now
-- an online concerts database, cross-referencing cities artists genres
venues, everything you could imagine, and sending weekly emails based
on your interests.
I got it up and running, pretty quickly, in php/mysql 3 years ago --
except it was ungodly slow. Part because of poor design on my fault,
part because of the tools I chose.
Two years ago, I recoded it into procedural perl and made a lot of
headway in optimizing it.
One year ago, I recoded it into object oriented perl under mod_perl,
and had even bigger gains in speed and use.
About the same time though, I started using Python a lot more. Now, I
rarely use anything else. So I'm trying to code it again, in twisted
and postgres, for a few reasons:
1 - i could touch perl a whole lot less, and deal with more
readable/editable code that i write way faster
2 - i didn't like the memory usage of what I had under mod_perl
(likely because of my errors, but twisted would be more strict and
make me keep in line) -- having a single daemon that handles all the
code just seems smarter and more efficient.
I'd be interested in your approach to caching. While the bulk of what
I see this project going is dynamic, certain elements and templates are
static -- and the constant reading off of disk seems excessive.
On Apr 15, 2004, at 7:51 PM, Stephen Thorne wrote:
> I'm going to give you straight answers, because I really don't believe
> in beating around the bush. I could give you curvy twisty statements
> that made you feel better about using twisted instead of apache while
> avoiding all figures, or I could lie to you with statistics by picking
> only the ones that are favourable.
> In short, twisted is slower than apache. This is a truth that you (and
> we) should acknowledge. Apache is a very mature web server that has
> stood the test of time. Its not the fastest and its not the best, but
> its pretty darn good.
> mod_php and mod_perl are blindingly fast. I can run simple mod_php
> scripts faster than i can serve static files with twisted.
> Twisted AND apache are still faster than the average connection.
> static files on a modest 800mhz machine gives me apache being twice as
> fast as twisted, but twisted was able to pump through 12meg/second. If
> you have a connection that exceeds 12meg/second transfers upstream, can
> you please contact me regarding getting an account.
> Serving woven was incredibly slow, so much so that with -n 1000 -c 100
> in ab (apachebench) I actually got a timeout. (I was serving a
> So, in face of all this? Should we use twisted?
> DEFINATELY. Twisted seriously cuts down on required development time
> achieving results with certain types of applications, especially
> applications that trancend the web model. Anything stateful, anything
> that wants to integrate with other protocols, anything that has long
> running processing (defer and pickup the result several page views
> later, YES PLEASE!).
> Also, a while back i did develop a caching mechanism for woven pages
> that allowed me to increase pageload speed by about 10fold. I think
> still got the code laying around somewhere, it was rather simple and
> very effective - based on the existing concept of woven models
> objects of when they change you could invalidate the cache and
> regenerate on the next pageload.
> Stephen Thorne.
> Twisted-web mailing list
> Twisted-web at twistedmatrix.com
More information about the Twisted-web