[Twisted-Python] twisted socksv4 proxy
bob at redivi.com
Wed Sep 24 00:47:39 EDT 2003
On Wednesday, Sep 24, 2003, at 00:08 America/New_York, Yun Mao wrote:
> I tried to use twisted python to write a proxy following the given
> socksv4 example.
> The problem i discovered is that: when i try to forward two
> their capacities may not match.
> In a blocking implementation,
> data = source.read(); dest.write(data);
> In a non-blocking implementation, what socksv4 did is something like:
> def dataReceived(data):
> However, if source is a much faster link than dest, would there be a
> problem? write always return even it should block so that read always
> happens, and the buffer of source tcp connection will never get full,
> let tcp start flow control?
With that code, you are right. Twisted will buffer everything for
dest.. so you won't lose any packets, but you might start using a lot
of memory and source may think that dest has a faster connection than
dest actually does.
The twisted.internet.policies module can throttle transports, however I
don't think that any of them can actually determine how fast dest is
actually consuming the data. That would require introspecting the
underlying transport's buffer and I don't believe the transport
interface has any way of doing that.
You could break the rules and assume that there is a dataBuffer
attribute (from twisted.internet.abstract.FileDescriptor) which would
most likely work, until you try and use it on a transport that is not a
FileDescriptor (wrapped by another policy, is a loopback for testing,
using a custom reactor, etc.).
In any case, yes, it's possible to solve this problem, but you will
have to write your own throttling policy which must bend the interface
rules in its implementation.
I would suggest writing the throttling policy code, and then posting
your code with a description of your use case to
http://twistedmatrix.com/bugs/ so there can be a discussion about how
to best change Twisted's interfaces around a bit so that this kind of
throttling policy can be implemented in a future release of Twisted
without digging into objects beyond their declared interfaces.
I could be mistaken, though. I'm not an expert on twisted.internet,
but I poked around it a bit when writing my own reactor.
More information about the Twisted-Python