[Twisted-Python] Throttling a web2 server
therve at free.fr
Wed Feb 27 05:19:59 EST 2008
Quoting Cameron Dale <camrdale at gmail.com>:
> On 2/26/08, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>> Yes. It would be good if ThrottlingFactory were fixed. Would you like
>> to fix it? :)
> Well, I took a stab at fixing it. Attached is my first attempt, as a
> patch against the current SVN version. I only fixed the write
> throttling, as that's the one I'm interested in. A similar technique
> may work for reads as well.
Thanks. Some maintenance on this would be great.
> It does throttle correctly, by buffering any throttled packets and
> pausing the producer. If the Sever then unpauses the producer, it will
> only result in an extra packet being buffered. I also implemented a
> protocol queue so that the producers are all given a fair portion of
> the bandwidth.
> The only thing it requires currently is that the producers don't try
> and write too large of a piece of data at once. I had to modify the
> twisted.web2.stream.FileSream slightly to make it do this, as
> otherwise it would 'stream' the entire file in one write() request. In
> theory, a single large write could be chunked and buffered, but that
> would add a lot of memory overhead to the buffering.
> Anyway, I'm sure it's not great, but I'd appreciate any feedback,
> especially on how I could make it better. :)
I haven't very closely at your patch, but there is definitely
something missing: tests. It's a requirement for anything going into
Twisted, and generally a requirement for making something works.
The discussion should probably continue in the tracker. You can open a
ticket, or maybe attach your patch at
http://twistedmatrix.com/trac/ticket/1460 (which doesn't exactly
describe the same goal).
More information about the Twisted-Python