[Twisted-Python] Throttling a web2 server

Thomas Hervé therve at free.fr
Wed Feb 27 05:19:59 EST 2008


Quoting Cameron Dale <camrdale at gmail.com>:

> On 2/26/08, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>> Yes.  It would be good if ThrottlingFactory were fixed.  Would you like
>>  to fix it? :)
>
> Well, I took a stab at fixing it. Attached is my first attempt, as a
> patch against the current SVN version. I only fixed the write
> throttling, as that's the one I'm interested in. A similar technique
> may work for reads as well.

Thanks. Some maintenance on this would be great.

> It does throttle correctly, by buffering any throttled packets and
> pausing the producer. If the Sever then unpauses the producer, it will
> only result in an extra packet being buffered. I also implemented a
> protocol queue so that the producers are all given a fair portion of
> the bandwidth.
>
> The only thing it requires currently is that the producers don't try
> and write too large of a piece of data at once. I had to modify the
> twisted.web2.stream.FileSream slightly to make it do this, as
> otherwise it would 'stream' the entire file in one write() request. In
> theory, a single large write could be chunked and buffered, but that
> would add a lot of memory overhead to the buffering.
>
> Anyway, I'm sure it's not great, but I'd appreciate any feedback,
> especially on how I could make it better. :)

I haven't very closely at your patch, but there is definitely  
something missing: tests. It's a requirement for anything going into  
Twisted, and generally a requirement for making something works.

The discussion should probably continue in the tracker. You can open a  
ticket, or maybe attach your patch at  
http://twistedmatrix.com/trac/ticket/1460 (which doesn't exactly  
describe the same goal).

-- 
Thomas






More information about the Twisted-Python mailing list