[Twisted-Python] twisted socksv4 proxy

Bob Ippolito bob at redivi.com
Wed Sep 24 00:47:39 EDT 2003


On Wednesday, Sep 24, 2003, at 00:08 America/New_York, Yun Mao wrote:

> I tried to use twisted python to write a proxy following the given
> socksv4 example.
>
> The problem i discovered is that: when i try to forward two 
> connections,
> their capacities may not match.
>
> In a blocking implementation,
>            data = source.read(); dest.write(data);
>
> In a non-blocking implementation, what socksv4 did is something like:
>
>    def dataReceived(data):
> 	otherConn.transport.write(data)
>
> However, if source is a much faster link than dest, would there be a
> problem? write always return even it should block so that read always
> happens, and the buffer of source tcp connection will never get full, 
> to
> let tcp start flow control?

With that code, you are right.  Twisted will buffer everything for 
dest.. so you won't lose any packets, but you might start using a lot 
of memory and source may think that dest has a faster connection than 
dest actually does.

The twisted.internet.policies module can throttle transports, however I 
don't think that any of them can actually determine how fast dest is 
actually consuming the data.  That would require introspecting the 
underlying transport's buffer and I don't believe the transport 
interface has any way of doing that.

You could break the rules and assume that there is a dataBuffer 
attribute (from twisted.internet.abstract.FileDescriptor) which would 
most likely work, until you try and use it on a transport that is not a 
FileDescriptor (wrapped by another policy, is a loopback for testing, 
using a custom reactor, etc.).

In any case, yes, it's possible to solve this problem, but you will 
have to write your own throttling policy which must bend the interface 
rules in its implementation.

I would suggest writing the throttling policy code, and then posting 
your code with a description of your use case to 
http://twistedmatrix.com/bugs/ so there can be a discussion about how 
to best change Twisted's interfaces around a bit so that this kind of 
throttling policy can be implemented in a future release of Twisted 
without digging into objects beyond their declared interfaces.

I could be mistaken, though.  I'm not an expert on twisted.internet, 
but I poked around it a bit when writing my own reactor.

-bob





More information about the Twisted-Python mailing list