[Twisted-Python] Packet Fragmentation

[glyph at divmod.com]

On 10:51 pm, nimag at rogers.com wrote:
>I was wondering if there's any way in Twisted/Python to handle packet 
>fragmentation.  Can I access the last fragment flag in the TCP packet 
>somehow? Or is there a total packet size sent with the packet 
>automatically?
>If not, what would be the best way to solve this?

What you're dealing with here is not packet fragmentation.  TCP packet 
fragmentation is a very specific phenomenon that occurs at a lower level 
than Twisted (and your application).

When you send a large message over a TCP stream, it can get broken up 
into multiple segments.  That's not the same as an individual packet 
getting fragmented.  There's no way to re-assemble the message to 
discover the original boundaries of the "write" call that you used to 
emit it.

In other words, your data stream needs to have message boundaries in it. 
TCP, at the application level, is a stream of octets, *not* a sequence 
of packets.

There are numerous examples of this in Twisted; for example, 
twisted.protocols.basic includes NetstringReceiver, LineReceiver and 
(various subclasses of) IntNStringReceiver, each of which is a different 
strategy for breaking up a TCP stream into a discrete sequence of 
messages.