[Twisted-Python] How to achieve a reliable "connection" over UDP?
glyph at divmod.com
glyph at divmod.com
Mon Jul 23 14:57:57 EDT 2007
On 09:23 am, p.mayers at imperial.ac.uk wrote:
>>I'm thinking basically all p2p applications will need this sort of
>>"reliability" over UDP for e.g. file transfer, so I'm hoping there is
>>something already written.
>Haven't used it, so can't comment further onit.
As it happens, I have, a little, so I can :).
To make a long story short, if you need the features that TCP provides,
but you want the simplicity (and widespread de-jure "standard"
implementation in routers) of hole-punching via UDP rather than complex
dual-connection TCP startup, the easiest thing you can do is just go
ahead and implement TCP on top of UDP.
We did this for vertex, in the "ptcp" module (pseudo-TCP) which is a
very literal-minded implementation of the TCP specification on top of
It's a fully-functional proof of concept at this point, but it needs a
lot of maintenance. The performance is terrible because we didn't
implement window sizing yet (the window has a static size), there are a
few edge-cases on connection tear-down, and the tests aren't quite
granular enough (for example, rather than testing specific latency
cases, they just probabilistically test packet loss). It also used a
predecessor to the now-standard-in-twisted "AMP" protocol for
handshaking, and should be upgraded to use the new one. Plus, my own
personal bugaboo: it doesn't integrate with cred properly.
However, if you need connection-oriented peer-to-peer traffic,
contributing to Vertex is going to get you where you need to be faster
than any ad-hoc solution. We *have* already implemented the full TCP
state machine and connection-management logic in Python. I've
personally tested that it can transfer an entire 700MB ISO image across
the real, public internet correctly, even in its current state.
You will find willing assistance from me and probably some other Divmod
folks & fans, because this is a project that many of us would like to
see through to completion; we just don't have a lot of spare cycles (or
business motivation) at the moment.
Depending on what you need, you might want to focus your maintenance
efforts on improving PTCP's tests and documentation so as to make it an
independently usable unit from Vertex's connection setup. Obviously
we'd like it if you could help us to get the whole thing to work, but I
have to assume that by the time you're asking this question, you've
already done some work on an application protocol to set up the
More information about the Twisted-Python