[Twisted-web] Streaming render

glyph at divmod.com glyph at divmod.com
Fri Sep 7 17:38:37 EDT 2007

On 03:01 pm, mithrandi-twisted-web at mithrandi.za.net wrote:
>* glyph at divmod.com <glyph at divmod.com> [2007-09-07 01:43:19 -0000]:
>>Use a recursive generator that yields Deferreds for loading each row.
>Thanks for the example! Combined with a DeferredQueue to pass the data
>from the callback to the rendering "loop", this approach seems to work

No problem!  Your Fan Club Dollars At Work!22.
>>Here's an example - which I even looked at in a web browser :) - that
>>should help to get you started.  (The athena stuff is beside the 
>>it's just the quickest way I know to get a server serving a Nevow 
>The athena-widget plugin is definitely very handy for quick examples
>like this.

We've also been discussing a slightly more general "twistd nevow" 
plugin.  More on this as it develops.
>>Warning: I haven't tested this on really large datasets, so it might 
>>disastrous performance consequences.  I don't *think* that it will, 
>>but I
>>know there are some booby-traps in the rendering pipeline to snare the
>As far as I can tell, nothing particularly nasty is happening when I 
>this; if the overhead of producing each row is very low, all the
>indirection through generators / deferreds might be costly, but that's
>certainly not the case for my code (the database and formatting
>operations are relatively expensive).

Good to know.
>I'm not really sure what I can do to drive the ticket along, but if
>there is something, please let me know.

I vaguely remembered this ticket when I wrote that comment :-).  Some of 
the things that are blocking it are on my list of stuff to get back to 
when I get back to Twisted development at all.

More information about the Twisted-web mailing list