[Twisted-web] reduce deferred stack in nevow

Valentino Volonghi aka Dialtone dialtone at divmod.com
Thu Jan 20 05:38:06 MST 2005

On Thu, 20 Jan 2005 04:09:17 +0100, Andrea Arcangeli <andrea at cpushare.com> wrote:

> Since you mentioned a performance problem, the real performance problem
> I have is in the rendering, that takes >150msec for my simplest page in
> the whole site. The slowest takes 800msec and I'm going to make some
> that will be even slower than that. You can easily reproduce with [0] *
> 300 passed as input to annotate.Choice(). Now for most of the http part
> a reverse proxy can probably do the trick, but for the ssl part it can't
> (even ignoring the ssl issues) because it's too dynamic.

> I wonder what can be done to boost the rendering (I was looking around
> the code, and I thought the addcallback/adderrback would be a cleanup
> but it wasn't sorry).

There are some issues to consider:

1) The optimizations branch will surely help a lot. I got a 2x speedup after using it (the branch caches every flattener lookup and does a great job in context lookups) unfortunately this is not yet merged and it's a bit old, so you may have to merge trunk with it first.

2) After that branch you may have pages that render in 70ms with the most complex that could take between 400-600ms.

Keep in mind that taking 70ms to render a page is not so slow for a single page. 

Another thing that could possibly work is rewriting the flatteners in Pyrex or even your page module in Pyrex, nothing stops you from that.

But first you should try to cache rendered fragments so that if data doesn't change you will be able to directly serve the pre-rendered html. 

And before everything you can use the load balancer, and once the session management will be improved you will also be able to share the session on many different servers.

More information about the Twisted-web mailing list