[Twisted-web] html cache with timeout

[Valentino Volonghi aka Dialtone]

On Tue, 1 Feb 2005 00:32:25 -0500, James Y Knight <foom at fuhm.net> wrote:
> Yes, it is. One thing that I think would make it more useful, if it can 
> be pulled off, is to allow caching at any level. That is, something 
> like the following stan:
> html[body[cached(timeout=10, keys=(IFoo, 
> IBar))[semi_constant_header_stuff], very_dynamic_content]]
> 
> That is, a function which will render its contents to a string the 
> first time it's called, and store/return it for the next 10 seconds, 
> using the same mechanism as the posted patch. The cache would be keyed 
> off certain interfaces, and only those will get passed on to the 
> functions being rendered inside, thus ensuring the safety of the cache.
> 
> I think there are a lot of pages that are mostly "semi-static", but 
> have some very dynamic content in them, so something like could be 
> *very* useful.
> 
> Anyhow, this is just an outline of how I think it might work, but I 
> haven't tried to implement it yet, so I don't know if it will work out 
> the way I'd like it to. :)

Here I am. Below there's a more fine grained implementation. You can use it like this:

cached(name="foo", lifetime=10)[t.p['hi boy']]

the name should be enough to give a cache for everyone since you can do:

cached(name=IAvatar(ctx).uid, lifetime=60)[t.p['Hi ', IAvatar(ctx).username]]

If you don't pass the lifetime parameter it won't be cached.

I must admit I haven't tested it yet since I don't know how to write unittests for this stuff... ;P. Let me know what do you think about. I also think that having another nevow tag for this cached would be ok.

Index: nevow/tags.py
===================================================================
--- nevow/tags.py       (revision 1136)
+++ nevow/tags.py       (working copy)
@@ -62,7 +62,9 @@
 def inlineJS(s):
     return script(type="text/javascript", language="JavaScript")[xml('\n//<![CDATA[\n%s\n//]]>\n' % s)]
 
-__all__ = tags + ['invisible', 'comment', '_dir', '_del', '_object', '_map', 'drange', 'Tag', 'directive', 'xml', 'raw', 'slot', 'cdata', 'inlineJS'] + ['_%s' % x for x in range(100)]
+__all__ = tags + ['invisible', 'comment', '_dir', '_del', '_object',
+                  '_map', 'drange', 'Tag', 'directive', 'xml', 'raw',
+                  'slot', 'cached', 'cdata', 'inlineJS'] + ['_%s' % x for x in range(100)]
 
 
 ########################
Index: nevow/flat/flatstan.py
===================================================================
--- nevow/flat/flatstan.py      (revision 1136)
+++ nevow/flat/flatstan.py      (working copy)
@@ -226,6 +226,31 @@
         return serialize(original.default, context)
     return serialize(data, context)
 
+_CACHE = {}
+from time import time as now
+def CachedSerializer(original, context):
+    if context.precompile:
+        original.children = precompile(original.children, context)
+        return original
+
+    cached = _CACHE.get(original.name, None)
+    if cached and cached[0] > now()-original.lifetime:
+        return cached[1]
+    toSerialize = serialize(original.children, context)
+    tmp = []
+    while 1:
+        try:
+            d = toSerialize.next()
+            tmp.append(d)
+        except StopIteration:
+            c = ''.join(tmp)
+            break
+        except AttributeError:
+            c = toSerialize
+            break
+    _CACHE[original.name] = (now(), c)
+    return c
+
 def ContextSerializer(original, context):
     originalContext = original.clone(deep=False)
     originalContext.precompile = context and context.precompile or False
Index: nevow/__init__.py
===================================================================
--- nevow/__init__.py   (revision 1136)
+++ nevow/__init__.py   (working copy)
@@ -182,6 +182,7 @@
 nevow.flat.flatstan.RendererSerializer            nevow.inevow.IRenderer
 nevow.flat.flatstan.DirectiveSerializer           nevow.stan.directive
 nevow.flat.flatstan.SlotSerializer                nevow.stan.slot
+nevow.flat.flatstan.CachedSerializer              nevow.stan.cached 
 nevow.flat.flatstan.ContextSerializer             nevow.context.WovenContext
 nevow.flat.flatstan.DeferredSerializer            twisted.internet.defer.Deferred
 nevow.flat.flatstan.DeferredSerializer            twisted.internet.defer.DeferredList
Index: nevow/stan.py
===================================================================
--- nevow/stan.py       (revision 1136)
+++ nevow/stan.py       (working copy)
@@ -119,8 +119,33 @@
         """
         raise NotImplementedError, "Stan slot instances are not iterable."
 
+class cached(object):
+    """Marker for cached content
+    """
+    __slots__ = ['name', 'children', 'lifetime']
 
+    def __init__(self, name, lifetime=0):
+        self.name = name
+        self.children = []
+        self.lifetime = lifetime
 
+    def __repr__(self):
+        return "cached('%s','%s')" % self.name, self.lifetime
+
+    def __getitem__(self, children):
+        """cached content is what is being cached
+        """
+        if not isinstance(children, (list, tuple)):
+            children = [children]
+        self.children.extend(children)
+        return self
+
+    def __iter__(self):
+        """Prevent an infinite loop if someone tries to do
+            for x in cached('foo'):
+        """
+        raise NotImplementedError, "Stan slot instances are not iterable."
+
 class Tag(object):
     """Tag instances represent XML tags with a tag name, attributes,
     and children. Tag instances can be constructed using the Prototype