[Twisted-Python] Large Transfers

Moshe Zadka m at moshez.org
Sat May 10 11:15:50 EDT 2003

On Sat, 10 May 2003, "Uwe C. Schroeder" <uwe at oss4u.com> wrote:

> in  another thread we had the "pb size limitation" issue. However it still 
> doesn't make too much sense to me, so probably someone of the core developers 
> can enlighten me.

The explanation is that the *other side* doesn't trust you -- maybe you're
trying to DoS it? This is a good design decision: you want multiple processes
to not trust each other. As an aside, whenever you hear the word "trust"
you should think "lack of security" -- by extension, "distrust" is "more

> Assume I have a remotecall using pb. According to the size limitation the 
> parameters given to a specific function can not exceed 640kb. What if I hand 
> over an object that exceeds the 640k ? Ok, the obvious happens and cBanana 
> throws an error "security precaution ....."

Yep. the cBanana on the other side decided that you looked like an evil
DoSer, and terminated you.

> Is this really a good thing to do ? Shouldn't pb see that the arguments are 
> larger than 640k and start paging ?

That's *way* to DWIMy, IMHO.

> What I'm doing is to hand down XML data which is database-generated on the 
> server side. Whenever a user requests a too large resultset the network layer 
> fails. On the other hand the resultset already is in memory, so why not jut 
> transfer it ?

Use StringPager. It's in memory. *Always* use StringPager, even if you're
below the security limit. 8k (by default) things will get transferred
in one go, larger strings will be sent in 8k chunks, allowing the other
side to digest them slowly without building big *intermediary* structures
in memory.

> So what makes is "more secure" to limit the size ??

That if your db-using process gets 0wned (if you pardon my trust),
it won't be able to drag down the other process.

Moshe Zadka -- http://moshez.org/
Buffy: I don't like you hanging out with someone that... short.
Riley: Yeah, a lot of young people nowadays are experimenting with shortness.
Agile Programming Language -- http://www.python.org/

More information about the Twisted-Python mailing list