[Twisted-web] load balancing and performance

[Valentino Volonghi aka Dialtone]

On Sun, 30 Jan 2005 12:46:35 +0100, Andrea Arcangeli <andrea at cpushare.com> wrote:
> So I believe it worth a try, and eliminating duplicated code sure cannot
> make things worse in the long run ;).

This is the biggest try though :). Formless will need a lot of work.
 
> > Index: rend.py
> > ===================================================================
> > --- rend.py     (revision 1105)
> > +++ rend.py     (working copy)
> > @@ -30,6 +30,7 @@
> >  from nevow import flat
> >  from nevow.util import log
> >  from nevow import util
> > +from nevow import url
> >  
> >  import formless
> >  from formless import iformless
> > @@ -376,6 +377,7 @@
> >              self.children = {}
> >          self.children[name] = child
> >      
> > +_CACHE = {}
> >  
> >  class Page(Fragment, ConfigurableFactory, ChildLookupMixin):
> >      """A page is the main Nevow resource and renders a document loaded
> > @@ -417,7 +419,8 @@
> >              io = StringIO()
> >              writer = io.write
> >              def finisher(result):
> > -                request.write(io.getvalue())
> > +                c = _CACHE[url.fromContext(ctx)] = io.getvalue()
> > +                request.write(c)
> >                  finishRequest()
> >                  return result
> >          else:
> > @@ -425,12 +428,17 @@
> >              def finisher(result):
> >                  finishRequest()
> >                  return result
> > +        c = _CACHE.get(url.fromContext(ctx), None)
> > +        if c is None:
> > +            doc = self.docFactory.load()
> > +            ctx =  WovenContext(ctx, tags.invisible[doc])
> > +            
> > +            return self.flattenFactory(doc, ctx, writer, finisher)
> > +        else:
> > +            request.write(c)
> > +            finishRequest()
> > +            return c
> >  
> > -        doc = self.docFactory.load()
> > -        ctx =  WovenContext(ctx, tags.invisible[doc])
> > -
> > -        return self.flattenFactory(doc, ctx, writer, finisher)
> > -
> >      def rememberStuff(self, ctx):
> >          Fragment.rememberStuff(self, ctx)
> >          ctx.remember(self, inevow.IResource)
> > 
> > This works and I've tested it.
> > 
> > Rendering speed went from 6-7 requests/sec to 26 req/sec on my poor ibook with the database on the same computer and ab too.
> 
> This is great, I'll play with this code very soon. This is a much more
> significant optimization than the load balancer, with the load balancer
> I could only double the number of pages per second.

Actually I wonder how did you manage to use this patch. I notice it is wrong in 2 lines when the code calls url.fromContext(ctx) which should be url.URL.fromContext(ctx).path