[Twisted-web] html cache with timeout

[Andrea Arcangeli]

On Tue, Feb 01, 2005 at 04:05:40PM +0000, Valentino Volonghi wrote:
> Anyway the patch is below:

Looks a great start.

I'll give it a spin overnight to see what happens.

> +_CACHE = {}

Shouldn't this be stored in the respective classes?

> +def CachedSerializer(original, context):
> +    cached = _CACHE.get(original.name, None)
> +    life = now()-original.lifetime

Can we execute only one single gettimeofday? gettimeofday is one of the
biggest kernel costs of twisted in general (modulo poll). I will deploy
initially on x86 (on x86-64 with vsyscalls gettimeofday is zerocost).

Could you also keep it similar to my patch where a timeout <= 0 means
"cache forever"?

> +    if cached and cached[0] > life:
> +##         print "="*20
> +##         print cached[0]
> +##         print life
> +##         print "="*20        
> +        yield cached[1]
> +        return

Why yield if you return immediatly? Why not return cached[1]?

> +    _CACHE[original.name] = (now(), result)

what is contained in original.name? How to identify exactly which object
is being cached? (just to understand how should I use this exactly)

> +    yield result

here again, why not return result?


Do I understand correctly this more finegriend cache doesn't obsolete
the other cache?

The other cache is probably the fastest we can get, and it pretty much
solves my problem for the high traffic part.

However I will definitely need this finegrined cache as well in the
longer run, even if it's lower priority. One of my first objectives
would be to workaround the dogslow rendering of the annotate.Choice
renderer. But I'll get great benefit in the header fragment and in other
fragments too.

As for the session cache, that's a much lower prio to me, the real
obvious caching we can do is for _global_ stuff that is the same for the
whole site, always.

Thanks and keep up the great work! ;)

PS. Still I would like to see compy removed, since not everything will
be cached. There are parts where I will not cache anything.