[Twisted-Python] Limit on transport.write

[Glyph]

On Mar 26, 2012, at 2:21 AM, hz hanks wrote:

> Hi, all
> 
> I'm writing a program to transfer files over Internet. Sometimes the
> files would be very big. Therefore, I'm not sure whether I could read
> a large block of data and send them via one time
> internet.protocol.transport.write(). On the other hand, does the
> function internet.protocol.transport.write() have any buffer scheme?
> If so, I could just read one line of the file each time and call that
> function. Or if you have any advice on how to transfer large amount of
> data, please let me know. Really appreciated.

First, you want to read this <http://twistedmatrix.com/documents/current/core/howto/producers.html>.

Second, you should understand that transport.write() will always buffer; it will never raise an error.  The rationale for this behavior is that if you've already got the data as a Python string (as you must, if you're calling write() with it), you have already paid the not-inconsiderable cost of pulling that data into your process, allocating memory for it, and slinging it around in Python, which probably means you've already copied it a few times by splitting it up, moving it around, etc.  The transport implementations will endeavor to not copy it around too much more (they generally keep a list of strings around as a buffer rather than a string they keep concatenating to), but they will hold on to it until they're able to write it.

If you want to know about the state of the buffer you need to subscribe to it using registerProducer() - so see the above document for how to do that.

-glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20120326/fbbd6e37/attachment.htm