[Twisted-Python] Is there a nice way to write transport-agnostic protocols and servers/clients?

[Cory Benfield]

> On 15 Aug 2017, at 01:22, Jarosław Fedewicz <jaroslaw.fedewicz at gmail.com> wrote:
> 
> The number of protocols that use TCP or UDP interchangeably is quite high. Some applications where Twisted would be an appropriate choice, could even work with non-TCP/UDP transports, like QUIC, DCCP, STCP, etc. 

Is it? TCP and UDP behave *very* differently: what protocols can safely use them interchangeably?

As to QUIC/DCCP/STCP and friends, ultimately the natural thing to do is to use composition of protocols, the same way TLSMemoryBIOProtocol does in Twisted. Essentially, you write a class (or collection of classes) that present themselves as both a protocol and a transport, and then you create a pipeline. For HTTP/2, for example, we have the following series of objects: reactor <-> TCP transport <-> TLSMemoryBIOProtocol <-> H2Connection <-> H2Stream <-> Request body handler protocol (provided by the user).

In each case, the intermediary objects provide both a transport and protocol interface. For example, from the TCP transport’s perspective, TLSMemoryBIOProtocol is a protocol. But from H2Connection’s perspective, it’s a transport. Similarly, H2Connection and H2Stream together provide both a protocol and transport interface: H2Connection is a protocol, H2Stream is a transport, and they communicate together.

For QUIC, ultimately it’s a protocol that runs over UDP. So you’d want to compose again: QUIC should be a protocol from the perspective of the UDP transport, and a transport from the perspective of its inner protocol (which would probably want to be something like HTTP, though there are some thorns here).

The only thing you can’t paste over is the difference between a streaming and non-streaming transport, which is as it should be: you cannot treat these two as identical. If *your specific protocol* can, then that’s ok: define an extra object that does the mapping. For example, imagine we’re using CorytextTransferProtocol, which can run over UDP and TCP equally well. Let’s not worry about how it does this (probably it has to reinvent TCP over UDP, but let’s not care). The way you’d do it is to define your core protocol logic in terms of, say, the stream transport interface (`class CorytextTransferProtocol` will call transport.write). Then, you write a shim class: `class UDPtoTCPforCTTPMapping`, say. This class does nothing if its transport is a stream transport, but does some appropriate transformation for datagram transports. Then, when you instantiate your protocol you set the mapping class as the protocol for the underlying transport, and then make the CorytextTransferProtocol class the protocol for the mapping. Essentially you get: underlying transport <-> UDPtoTCPforCTTPMapping <-> CorytextTransferProtocol.

The great advantage of this is that your two classes can be decoupled, so if the strategy of mapping streaming to datagram transport is general it can be re-used by other protocols that want a streaming interface.

Does this make sense?