[Twisted-web] Re: html cache with timeout

[Andrea Arcangeli]

On Tue, Feb 01, 2005 at 10:21:22PM -0500, Alex Levy wrote:
> I think the aim should be to make this caching feature sufficiently useful
> on its own, while being sufficiently extensible to allow people to do what
> they want with it. It'd be nice to have the option of using memcached, or an
> in-process cache, or... well, you get the idea.
> 
> I liked Matt's idea of storing something in the context, like an
> ICacheManager. Of course, I should disclose I liked the idea because I'd
> planned on the same thing for Payago. :)

Sure, I'm not against options.

> Of course, I think caching is one of those problems that will always be very
> site-specific, and any implementation that involves modifying the Page class
> is going to be either too general-purpose to be useful, or too invasive to
> be stable all the time, for all projects. Your httprender patch is useful,

That's why it can be enabled on a page-by-page basis. And I get 100%
caching of the whole http site with the pure cache in rend.Page and it
has taken me minutes to enable it and now it runs at 200req/sec instead
of 7req/sec. If I change it, I will get no benefit, the API will be more
complex, and I only risk to run slower.

All forms I have are under ssl, for the ssl part I could pratically
cache nothing with that cache and I need the more finegrined one (that I
already successfully enabled on all forms but one that is so dynamic
that I can't cache it even with the stan cache, but luckily it's the one
for the account registration, so it's sure not high traffic).

If you could give me a way to use the finegrined cache to cache a whole
xml page, then I could benchmark the difference and evaluate if perhaps
I should take the pain of using the more complex less ideal API for the
http part just to reduce the nevow complexity and to avoid changing
rend.Page. I never use loader.stan for the http part (and that's the
part I can easily benchmark since it has no guard and no ssl).

> but probably better placed in a subclass.

That would probably duplicate code, so I don't think it's ideal, even if
it would save you a check on self.cacheTimeout per rendering if you
don't use it.