[Twisted-web] html cache with timeout

[Andrea Arcangeli]

On Wed, Feb 02, 2005 at 07:56:14AM +0000, Valentino Volonghi wrote:
> The benefits are that it scales to multiple computers, which means a
> better way to distribute the load. But, as I said, it's just one of
> the possible backends.

For most common sites, I doubt the gain of scaling the caching could
ever be significant. There's so much memory on each system that local
caching w/o context switches and w/o interprocess communication makes
more sense IMHO. Especially if you can get to the data in almost
constant time with hashes. I get a 4/5msec total time for each page with
local caching in the httprender, it would probably get at visible
slowdown if I were to access the cache remotely through sockets.

Anyway I'm not against options, infact I'd like to keep the option
myself to use the so handy and trivial to use httprender cache ;), I'm
just prioritizing on what's is more important first ;)

> In weever I did a test, and I simply put this in the base class where
> I fill the content slot (each page is a main page which contains a
> content slot that is filled with the content Fragment):
> 
> t.cached(name=str(self.__class__)+IA(ctx).get('uid', ''), lifetime=10)[...]

Isn't that going to leak memory badly with tons of users seldom
accessing the site? At least for my usage I can't do the above or it
would risk to run my app out of memory.  lifetime isn't a timer, so
it'll never be freed.

To get automatic garbage collection one should store it in the
session, or at least attach a timer that fires indipendently if the
rendering is enabled or not.

> Doing this in the main baseclass gave me caching on ALL pages with a per/user cache.
> Anyway this is just an implementation problem. Nobody stops you from doing:
> 
> def myHttpCache(ctx, lifetime):
>     return t.cached(name=str(url.URL.fromContext(ctx)), lifetime=lifetime)
> 
> Which will give you the same result.

I'm using this only for the forms right now:

def renderCachedForms(ctx, lifetime=0, *args, **kwargs):
	return tags.cached(lifetime=lifetime,
			   name='formless-'+str(url.URL.fromContext(ctx)))[webform.renderForms(*args, **kwargs)]

It's very handy to use by just repalcing webform.renderForms() with
renderCachedForms(ctx).

> When I say that it won't work with xml templates I mean that you need
> a way to create a cached tag from the xml. This is all.

I still doubt it'll be as fast as httprender cache, but I can try.

It has taken me minutes to enable httprender caching in the whole http
site, so I tend to consider that API more handy, and peformance should
be a bit higher too. So I don't see much point to use the stan cache in
order to cache whole documents when the other lowlevel cache can be used
in the rend.Page.