[Twisted-Python] Throttling a web2 server

Cameron Dale camrdale at gmail.com
Wed Feb 27 00:20:17 MST 2008


On 2/26/08, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> Yes.  It would be good if ThrottlingFactory were fixed.  Would you like
>  to fix it? :)

Well, I took a stab at fixing it. Attached is my first attempt, as a
patch against the current SVN version. I only fixed the write
throttling, as that's the one I'm interested in. A similar technique
may work for reads as well.

It does throttle correctly, by buffering any throttled packets and
pausing the producer. If the Sever then unpauses the producer, it will
only result in an extra packet being buffered. I also implemented a
protocol queue so that the producers are all given a fair portion of
the bandwidth.

The only thing it requires currently is that the producers don't try
and write too large of a piece of data at once. I had to modify the
twisted.web2.stream.FileSream slightly to make it do this, as
otherwise it would 'stream' the entire file in one write() request. In
theory, a single large write could be chunked and buffered, but that
would add a lot of memory overhead to the buffering.

Anyway, I'm sure it's not great, but I'd appreciate any feedback,
especially on how I could make it better. :)

Thanks,
Cameron
-------------- next part --------------
A non-text attachment was scrubbed...
Name: policies.py.patch
Type: text/x-diff
Size: 7325 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20080226/1b95844b/attachment-0002.patch>


More information about the Twisted-Python mailing list