[Twisted-Python] twistedcat - twisted endpoint concatenator

[Glyph]

On Jun 21, 2014, at 7:12 AM, David Stainton <dstainton415 at gmail.com> wrote:

> Greetings,
> 
> I was inspired by the "socat" CLI utility to write something similar called twistedcat.
> 
> https://github.com/david415/twistedcat
> https://github.com/david415/twistedcat/blob/master/twistedcat/twistedcat.py
> 
> Obviously it should have the "cat" word in the name... so what do you suggest?

How about "nocat" (network oriented cat), in honor of <http://quoteinvestigator.com/2012/02/24/telegraph-cat/>?

> What do people think of this?
> Do you have ascii art drawings of cats can use or suggestions for me to improve the code quality?

Are you saying that you have internet access, but cannot find pictures of cats? :-)

> Right now I'm cleaning the code out of main() and creating an EndpointCombiner class
> to encapsulate the endpoint pipe logic (c2c, s2s, c2s).
> 
> The reason I am very interested in improving the code to assist in pipe-lining endpoints is because it can be used by more than just this commandline tool. Any Twisted app wishing to create custom protocols can do so by pipe-lining endpoints. I think the API for this could be very easy to use.

We do have some work going in to making a generic wrapper-endpoint facility so it's easier to add wrapping to endpoints.  The branch for <https://twistedmatrix.com/trac/ticket/5642#comment:28> has such features.

> socat has an enormous feature set that I am not sufficiently motivated to duplicate here.

Yes, I think it would be better to focus on the alternate/awesome features that Twisted already gives you, rather than aiming for parity.

> http://www.dest-unreach.org/socat/doc/socat.html#ADDRESS_TYPES
> 
> It was really interesting for me to read about these socat address types... there are many of them.
> Twisted has some of the same types... and in some cases they are even spelled the same.
> I am interested in implementing some of these socat address types in Twisted, including a TUN endpoint and endpoint parser plugin.
> I've played around with the new twisted pair TUN stuff... but I was using producers and consumers to make a VPN...
> and as it turns out this task should be greatly simplified by using pipelined Twisted endpoints.

Well, this all sounds amazing to me.  Please keep us posted!

> I've also been toying with the idea of writing an obfsproxy Twisted endpoint which would work similar to the txtorcon endpoint in that it would launch an external command... and then perform a tcp connect. obfsproxy can be launched in external mode and has several obfuscation protocols to choose from... some of which are very sophisticated. If someone actually had a good use case for obfuscating pipelined endpoint protocols then I would love to hear about it (besides the obvious VPN use-case)!
> 
> https://www.torproject.org/projects/obfsproxy
> 
> Twisted endpoint descriptor strings are just like socat address specifications... except that in socat the client and server endpoint types are in the same name space. Therefore twistedcat needs to know if the endpoint string is a server or client endpoint. It defaults to client unless -s or --server specifies it as a server endpoint string. Or do you dare suggest that it would be better to try clientFromString and then if that fails try serverFromString?

Personally I think having a -s option (or -l to follow the tradition set forth by nc?) to be explicit is better.

Have you considered making wrappers in both directions though: a string-parsing plugin that turns a server endpoint into a client endpoint by making .listen() call connect, and one that turns a client endpoint into a server endpoint by having .connect() listen and wait for the first inbound connection?

> We've already created a Tor client endpoint/parser plugin (https://github.com/david415/txsocksx)
> and a Tor Hidden Service endpoint/parser plugin (https://github.com/meejah/txtorcon)...
> Obviously twistedcat doesn't know anything about Tor... because it's completely endpoint agnostic.
> 
> Here's my silly twistedcat Tor experimentations:
> 
> $ ./twistedcat.py -s stdio: -s onion:80:hiddenServiceDir=/home/human/hs
> 
> After running this one liner... I checked the contents of /home/human/hs/
> and found my new onion address in the "hostname" file.
> I used my Tor browser to send a request... and here we see twistedcat printed it to stdout via the stdio endpoint:
> 
> GET / HTTP/1.1
> Host: ibce3yvjdb3mzcrq.onion
> User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Firefox/24.0
> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language: en-us,en;q=0.5
> Accept-Encoding: gzip, deflate
> Connection: keep-alive
> 
> 
> And then there is this:
> 
> $ ./twistedcat.py tor:timaq4ygg2iegci7.onion:80 -s onion:80:hiddenServiceDir=/home/human/hs
> 
> Which of course proxies requests from my new onion to timaq4ygg2iegci7.onion... the onion address of the txtordoc documentation website.
> But this is crazy... um how many hops is that exactly? 14? 15?

Sounds like fun to me :-).

-glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20140621/73d72e11/attachment.html>