[Twisted-web] Cache-friendly Nevow pages

[James Y Knight]

On Aug 8, 2004, at 1:26 AM, Mary Gardiner wrote:

> On Sat, Aug 07, 2004, Mary Gardiner wrote:
>>  2. generate the ETag header based on a hash of page contents
>>
>>    Pros: As best I can tell, this is how the ETag header is really 
>> meant
>>    to be generated, ideally it signals octect equality and should 
>> change
>>    if, for example, Nevow for some reason starts pretty-printing 
>> output.
>>
>>    Cons: rend.Page.renderHTTP seems to make this really hard --
>>    even if you set the bufferedflag = True, rend.Page.afterRender
>>    doesn't seem to have any way to access the result of the render.
>
> The attached seems to do what I want here, I'll be interested to hear
> about what horrors I am inflicting upon myself by doing this.

Generally, ETags are never actually generated by hashing the page 
content. That's just too expensive. To do that, you need to actually 
generate the content before you can tell that you shouldn't send it to 
the user. What a waste of resources!

Apache (and new-web) generate ETags from files via: ETag("%X-%X-%X" % 
(st.st_ino, st.st_size, st.st_mtime), weak = (time.time() - st.st_mtime 
<= 1))

That "good enough" guarantees that the file still has the same content 
(the etag is weak if the last-modified is recent, because the file 
could be modified twice within the last second and then you wouldn't 
know).

So, for dynamically generated pages, I'd do something similar (e.g. 
your #1). Not sure what you mean about about reducing flexibility or 
what you mean about it making you need to restart the server. If you 
have to restart the server every time you change a template (e.g. for 
stan), use the server start time as part of the ETag. Otherwise, use 
the template modtime. The file-based doc-factories do store mtime, 
although it's currently in a private attribute. That will get updated 
when the template is reloaded.

James