[Twisted-Python] Is there a nice way to write transport-agnostic protocols and servers/clients?

Jarosław Fedewicz jaroslaw.fedewicz at gmail.com
Wed Aug 16 05:53:07 MDT 2017


This makes a perfect sense. (I ended up implementing it in this sorta way,
but the other way around: it was a primarily-datagram protocol that could
use TCP as a transport, provided there is appropriate framing.)

However, while there is really a need for a shim to make use of
polytransport protocols (DNS, syslog, SIP, NFS are a few), it looks like
every Twisted user should bring their own. This bothers me, but just a
little bit.

Also, your reply does not address the little discrepancies that exist in
Twisted between two kinds of datagram protocols (namely, Unix datagram
sockets and UDP): seems like UDP APIs progressed while Unix datagram socket
support was "left behind". I don't know if this is deliberate or not.

One thing that is also notably missing from the puzzle is that there is no
support for datagram endpoints. With stream sockets, I can now use strings
and clientFromString()/serverFromString() and be blissfully ignorant if
it's TCP or a Unix socket. I cannot do the same with datagram sockets, I
need to be actively aware if what I'm dealing with is a UDP socket or a
Unix datagram socket.

I'd gladly implement something to fill the gap, but I fear my contribution
would be rejected because, say, it doesn't conform to the general direction
Twisted is heading, whatever that may be. Or because datagram protocols are
somehow inferior as a class. Or because Unix datagram sockets in particular
should die and burn in hell.



On Wed, Aug 16, 2017 at 10:07 AM, Cory Benfield <cory at lukasa.co.uk> wrote:

>
>
> > On 15 Aug 2017, at 01:22, Jarosław Fedewicz <jaroslaw.fedewicz at gmail.com>
> wrote:
> >
> > The number of protocols that use TCP or UDP interchangeably is quite
> high. Some applications where Twisted would be an appropriate choice, could
> even work with non-TCP/UDP transports, like QUIC, DCCP, STCP, etc.
>
> Is it? TCP and UDP behave *very* differently: what protocols can safely
> use them interchangeably?
>
> As to QUIC/DCCP/STCP and friends, ultimately the natural thing to do is to
> use composition of protocols, the same way TLSMemoryBIOProtocol does in
> Twisted. Essentially, you write a class (or collection of classes) that
> present themselves as both a protocol and a transport, and then you create
> a pipeline. For HTTP/2, for example, we have the following series of
> objects: reactor <-> TCP transport <-> TLSMemoryBIOProtocol <->
> H2Connection <-> H2Stream <-> Request body handler protocol (provided by
> the user).
>
> In each case, the intermediary objects provide both a transport and
> protocol interface. For example, from the TCP transport’s perspective,
> TLSMemoryBIOProtocol is a protocol. But from H2Connection’s perspective,
> it’s a transport. Similarly, H2Connection and H2Stream together provide
> both a protocol and transport interface: H2Connection is a protocol,
> H2Stream is a transport, and they communicate together.
>
> For QUIC, ultimately it’s a protocol that runs over UDP. So you’d want to
> compose again: QUIC should be a protocol from the perspective of the UDP
> transport, and a transport from the perspective of its inner protocol
> (which would probably want to be something like HTTP, though there are some
> thorns here).
>
> The only thing you can’t paste over is the difference between a streaming
> and non-streaming transport, which is as it should be: you cannot treat
> these two as identical. If *your specific protocol* can, then that’s ok:
> define an extra object that does the mapping. For example, imagine we’re
> using CorytextTransferProtocol, which can run over UDP and TCP equally
> well. Let’s not worry about how it does this (probably it has to reinvent
> TCP over UDP, but let’s not care). The way you’d do it is to define your
> core protocol logic in terms of, say, the stream transport interface
> (`class CorytextTransferProtocol` will call transport.write). Then, you
> write a shim class: `class UDPtoTCPforCTTPMapping`, say. This class does
> nothing if its transport is a stream transport, but does some appropriate
> transformation for datagram transports. Then, when you instantiate your
> protocol you set the mapping class as the protocol for the underlying
> transport, and then make the CorytextTransferProtocol class the protocol
> for the mapping. Essentially you get: underlying transport <->
> UDPtoTCPforCTTPMapping <-> CorytextTransferProtocol.
>
> The great advantage of this is that your two classes can be decoupled, so
> if the strategy of mapping streaming to datagram transport is general it
> can be re-used by other protocols that want a streaming interface.
>
> Does this make sense?
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>



-- 
Yaroslav Fedevych
IT Philosopher
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20170816/5ed73914/attachment-0002.html>


More information about the Twisted-Python mailing list