[Twisted-Python] Status of current proxy support (of various kinds)

Glyph Lefkowitz glyph at twistedmatrix.com
Tue Aug 9 00:31:19 EDT 2011

On Aug 8, 2011, at 10:10 PM, Peter Le Bek wrote:

> Myself and another are working on adding SOCKS client support to Twisted for 
> use in Tor related projects. I'm also convinced there's an abstraction that would 
> make it easier to implement proxy clients (or any transparent data/endpoint 
> modifying protocol) in Twisted. I agree that it's difficult to generalise, but there's a 
> class of protocol that does one or more of the following, transparently, and little 
> else:
> - append data, before switching to application protocol (SSL handshake, SOCKS connect/bind)
> - prepend data, after losing connection on application protocol (SSL shutdown)
> - modify application data (SSL "recordification", encryption)
> - modify the endpoint (any proxy)
> Endpoints can do all these things - SSL4ClientEndpoint, for example, does the first 
> three - but does it make sense to use Endpoints for this purpose? 

I think so.

> Endpoints aren't stackable, so no good if I want to run a SOCKS client over SSL.

Sure they are.  There's no SOCKS endpoint yet, but if you were to implement one, you could make it take another endpoint (or endpoint factory, as the case may be) as an argument to its constructor.  The implementation of the various current SSL* enpdoints would set a better example if it weren't for the legacy connectSSL/listenSSL APIs.  However, those APIs are being slowly phased out in favor of the endpoints themselves, as well as the internal implementation being moved to use a wrapping / delegation approach <http://twistedmatrix.com/trac/ticket/4854> between transports and protocols instead of OpenSSL's SSL-is-a-socket-except-when-it-isn't strategy.

> Also, with a proxied connection you have two Endpoints, one to the proxy server 
> and one 'virtual Endpoint' to wherever. The Endpoints API doesn't accommodate 
> this.

How doesn't it?  It's exactly this kind of API that endpoints were _designed_ to accommodate - the original impetus to create them was the Vertex project (now part of <https://launchpad.net/divmod.org/>), which proxies traffic in a variety of different ways.

> Would it make sense to have Endpoint wrappers for this purpose? A sort of 
> middleware (hate that word) to intercept transport events (including transport 
> formation, i.e. Endpoint.connect()).

It might make sense to have some utilities around this area.  There are definitely parts of the process - especially producers and consumers - where you have to write a lot of boilerplate code to make sure everything's hooked up properly.  But it's all certainly _possible_ now with the existing interfaces.

You'd probably be interested in this ticket: <http://twistedmatrix.com/trac/ticket/3204>.  Perhaps you could submit a patch? :)

> I'm looking forward to being proven wrong on this as it will make my SOCKS work 
> easier, or else if this turns out to be interesting then I have more ideas on how it 
> should be implemented.

It would help with that proof if you would be a bit more precise about where you think the current code falls short.


More information about the Twisted-Python mailing list