[Twisted-web] load balancing and performance

Valentino Volonghi aka Dialtone dialtone at divmod.com
Sat Jan 29 17:26:33 MST 2005

On Sat, 29 Jan 2005 15:55:50 +0100, Andrea Arcangeli <andrea at cpushare.com> wrote:
>I successfully load balanced the static server with pythondirector.
> However it has a few problems:
> 1) it cannot handle ssl
> 2) even if it can handle ssl I still need to share the session cookies
>    and I'm thinking that using sql or atop to do that isn't necessary:
>    the session doesn't need to survive a reboot, a reboot would be a visible
>    disruption of the services anyway, so I'm thinking to write a little
>    session daemon that only stores the cookies using pb to do that.
>    In theory shared memory would be much more efficient in smp, but at least
>    this socket model has the advantage of scaling in the cluster too
> 3) the load balancer misses an API with the real webserver to pass up the
>    client IP address, that's annoying, especially the logs gets screwed
> Other than that the speed of the load balancer is excellent, and I get
> exactly the double number of pages rendered per second (after fixing a
> small bug in pythondirector that confused ab2).

It's probably also a good idea to write a balancer that works on unix sockets. And this means also writing a good path to which it should dispatch. 

Tell me more about the session daemon. Anyway we are designing an ISessionManager interface to let you write whatever sessionFactory you need, a database or a SessionDaemon or a file or something else. Probably you can help with it by coming in #twisted.web and commenting it with one of us (Donovan, Matt, Tv, me and others). 
> Perhaps I'm going with wrong priorities though, the major offender is
> compy, compy must be dropped from Nevow ASAP :). Leaving it as a

compy is not going away :). Writing a compy2 speedup in Pyrex will help and will probably also be faster than zope.interface since it will be a lot smaller.

> compatibility API may be ok, but internally compy can't invoke
> getComponents anymore if we do care about writing remotely optimal code.
> We should try with the new zope.interfaces API first and see if it
> underperforms so horribly as getInterfaces does.

zope.interface is twice as fast without the compatibility stuff in twisted and I think it is the same for Nevow.

> Secondly I'm looking into caching the html and to render some fragment only
> once every 10 seconds in the background (so the downloads will never
> have to wait for a rendering of some mostly static fragment anymore).

I wrote this VERY simple stuff for caching a page:

Index: rend.py
--- rend.py     (revision 1105)
+++ rend.py     (working copy)
@@ -30,6 +30,7 @@
 from nevow import flat
 from nevow.util import log
 from nevow import util
+from nevow import url
 import formless
 from formless import iformless
@@ -376,6 +377,7 @@
             self.children = {}
         self.children[name] = child
+_CACHE = {}
 class Page(Fragment, ConfigurableFactory, ChildLookupMixin):
     """A page is the main Nevow resource and renders a document loaded
@@ -417,7 +419,8 @@
             io = StringIO()
             writer = io.write
             def finisher(result):
-                request.write(io.getvalue())
+                c = _CACHE[url.fromContext(ctx)] = io.getvalue()
+                request.write(c)
                 return result
@@ -425,12 +428,17 @@
             def finisher(result):
                 return result
+        c = _CACHE.get(url.fromContext(ctx), None)
+        if c is None:
+            doc = self.docFactory.load()
+            ctx =  WovenContext(ctx, tags.invisible[doc])
+            return self.flattenFactory(doc, ctx, writer, finisher)
+        else:
+            request.write(c)
+            finishRequest()
+            return c
-        doc = self.docFactory.load()
-        ctx =  WovenContext(ctx, tags.invisible[doc])
-        return self.flattenFactory(doc, ctx, writer, finisher)
     def rememberStuff(self, ctx):
         Fragment.rememberStuff(self, ctx)
         ctx.remember(self, inevow.IResource)

This works and I've tested it.

Rendering speed went from 6-7 requests/sec to 26 req/sec on my poor ibook with the database on the same computer and ab too.

This patch is simple, probably too simple (in fact it would be better to cache the flattening result, this would be a lot more fine grained) since it only works in buffered mode (patching this patch to work in non buffered mode is not hard at all though)

> So overall there's an huge room for improvements. What do other people
> think?

I also think that the optimizations branch is worth of some experimentation. I got twice the rendering speed in dynamic pages thanks to its adapters caching. I'd give it a try.

Overall I think nevow can, and will, speedup at least by a factor of 5.

More information about the Twisted-web mailing list