[Twisted-Python] Endpoint composition syntax

[meejah]

Tristan Seligmann <mithrandi at mithrandi.net> writes:

> Currently there is no way to explicitly compose Twisted endpoints, but
> several endpoint implementations have arisen that explicitly wrap
> another endpoint, and so have needed a way to do this.

A couple other examples:

Autobahn provides 'Web Application Messaging Protocol' (WAMP) endpoints
that can use either a Unix, TCP or WebSockets protocol under the hood --
having a proper endpoint syntax for this would be nice. There is *some*
support for endpoint-strings in Autobahn using the backslash trick, but
this also results in some ugliness like:

   r"autobahn:tcp\:9000\:interface\=0.0.0.0:url=ws\://localhost\:9000:compress=false"
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^     ~~~~~~~~~~~~~~~~~~~~~

Ideally txtorcon would have a way to specify "how" to connect for
client-side connections meaning "I want a stream over the Tor whose
control-port is 'unix:/foo' that connects via SOCKS at unix:/bar to
https://meejah.ca". (This can only be done in code, currently, not
endpoint-strings) Now think about specifying that you want the above
Autobahn connection to go over Tor ;) that is, replacing
"https://meejah.ca" with the monster above...

     *   *   *

On a possible tangent: I wonder if this also ties in with trying to wrap
protocols to "hand off" their transport to another one? Examples of this
are: SOCKS (e.g. speaking one protocol "after" another), or in Autobahn
where it's nice to listen for both "normal" Web requests and also
WebSockets requests on the same port (so there's a protocol that "peeks"
and hands off to HTTP or WebSockets handlers).

I can also imagine doing the same thing with http://magic-wormhole.io
where you would establish a connection via the wormhole mechanism, and
then pass over the established transport to the "real" protocol
(e.g. could be HTTP, WAMP, SSH, WebSockets, something custom, etc) and
this at least has an obvious need for a corresponding string-parser
syntax like the one suggested.

> tls:awesome.site.example.com:443->tcp:7.6.5.4:443 A less whimsical
> syntax than "->" might be better; for example, semicolons, or
> something like that.

I wonder if a simple space would work? Downsides would be: requiring
quoting on shells; maybe it would present problems in config-file
use-cases; ...

So, we want "something" the highest-level parser can split on before
handing off bits (or all) of it to the actual plugins:

 - can't have ":" since that's already a separatator
 - can't have "=" since it's already used
 - shouldn't already be in popular protocols/options (e.g. anything
   valid in a URI?)

In some ways having a two-character separator could be really nice, as
it's far less likely to collide with things? I like that the intent of
"->" is also reasonably obvious, I think. Or at least hopefully looks
strange enough that you'll look it up ;)

Thanks for kicking off discussion :)

-- 
meejah