[Twisted-Python] Sending a stream via Protocol?

[exarkun at twistedmatrix.com]

On 04:42 pm, darren at ontrenet.com wrote:
>Hi,
>  I have a simple protocol listening on a port. I send it a text message
>(a filename)
>and the protocol tries to stream back the bytes of the file. Mostly,
>this works
>but in some cases, the receiving side receives less bytes than is
>written by:
>
>self.transport.write(bytes)
>
>I read the API which says it can receive less than expected bytes or
>messages. Why?

That's how TCP works.  Bytes you send with one call to write may be 
split into two or more chunks and delivered to the remote dataReceived 
separately, or bytes you send with two or more calls to write may be 
combined into fewer chunks and delivered to the remote dataReceived all 
as one string.
>
>So how can I reliably send a sequences of bytes via "dataReceived" in 
>my
>protocol?

If you want "framing" - a byte format built on top of the basic 
functionality provided by TCP, allowing you to differentiate bytes 
belonging to different logical messages from each other - then you 
should take a look at some of the protocols in twisted.protocols.basic 
(NetstringReceiver, Int32StringReceiver, etc) or twisted.protocols.amp 
which expands on this idea to a much greater degree.

You can read a lot more about these ideas in this series of articles by 
Itamar Turner-Trauring:

  http://www.xml.com/pub/au/215

Jean-Paul