[Twisted-Python] Large Transfers

Bernhard Herzog bh at intevation.de
Sat May 10 12:01:40 EDT 2003

Moshe Zadka <m at moshez.org> writes:

> On Sat, 10 May 2003, "Uwe C. Schroeder" <uwe at oss4u.com> wrote:
> > in  another thread we had the "pb size limitation" issue. However it still 
> > doesn't make too much sense to me, so probably someone of the core developers 
> > can enlighten me.
> The explanation is that the *other side* doesn't trust you -- maybe you're
> trying to DoS it? This is a good design decision: you want multiple processes
> to not trust each other. As an aside, whenever you hear the word "trust"
> you should think "lack of security" -- by extension, "distrust" is "more
> secure".

ISTM that this approach doesn't really give you much security. It only
helps against accidentally large chunks of data.

The tests, AFAICT, simply check the length (as defined by the len
builtin) of lists, tuples and strings. So if the sender can't send e.g.
[0] * 1000000, it could just as well send [[0] * 1000 for i in range(1000)]
which would consume roughly the same amount of memory on the receiver's
side but would pass all the SIZE_LIMT tests. Using a large dictionary
seems to work too.

A better solution for limiting the size seems to be a limit on the
number of bytes accepted by the receiver for a single PB call.

Anyway, limits on the data size should be configurable IMO so that
programmers can set the limit based on the needs of their applications.

Finally, forcing users of PB to use a different protocol for large
datasets doesn't improve matters. If a malicious user can exploit the PB
connection they can also exploit this other protocol which of course has
no size limit because that's the reason to use it in the first place.


Intevation GmbH                                 http://intevation.de/
Sketch                                 http://sketch.sourceforge.net/
MapIt!                                           http://www.mapit.de/

More information about the Twisted-Python mailing list