[Twisted-Python] twisted socksv4 proxy

Yun Mao maoy at cis.upenn.edu
Tue Sep 23 23:16:06 MDT 2003


Is it true that the only way to see the success of write() is to check the 
FileDescriptor.dataBuffer? 

I'm thinking that suppose there is one way to check this, and I can
somehow add a callback function to the success of each write(), then I can
simply call source.stopReading() after write() and startReading() when I
get callback.

Yun


On Wed, 24 Sep 2003, Bob Ippolito wrote:

> On Wednesday, Sep 24, 2003, at 00:08 America/New_York, Yun Mao wrote:
> 
> > I tried to use twisted python to write a proxy following the given
> > socksv4 example.
> >
> > The problem i discovered is that: when i try to forward two 
> > connections,
> > their capacities may not match.
> >
> > In a blocking implementation,
> >            data = source.read(); dest.write(data);
> >
> > In a non-blocking implementation, what socksv4 did is something like:
> >
> >    def dataReceived(data):
> > 	otherConn.transport.write(data)
> >
> > However, if source is a much faster link than dest, would there be a
> > problem? write always return even it should block so that read always
> > happens, and the buffer of source tcp connection will never get full, 
> > to
> > let tcp start flow control?
> 
> With that code, you are right.  Twisted will buffer everything for 
> dest.. so you won't lose any packets, but you might start using a lot 
> of memory and source may think that dest has a faster connection than 
> dest actually does.
> 
> The twisted.internet.policies module can throttle transports, however I 
> don't think that any of them can actually determine how fast dest is 
> actually consuming the data.  That would require introspecting the 
> underlying transport's buffer and I don't believe the transport 
> interface has any way of doing that.
> 
> You could break the rules and assume that there is a dataBuffer 
> attribute (from twisted.internet.abstract.FileDescriptor) which would 
> most likely work, until you try and use it on a transport that is not a 
> FileDescriptor (wrapped by another policy, is a loopback for testing, 
> using a custom reactor, etc.).
> 
> In any case, yes, it's possible to solve this problem, but you will 
> have to write your own throttling policy which must bend the interface 
> rules in its implementation.
> 
> I would suggest writing the throttling policy code, and then posting 
> your code with a description of your use case to 
> http://twistedmatrix.com/bugs/ so there can be a discussion about how 
> to best change Twisted's interfaces around a bit so that this kind of 
> throttling policy can be implemented in a future release of Twisted 
> without digging into objects beyond their declared interfaces.
> 
> I could be mistaken, though.  I'm not an expert on twisted.internet, 
> but I poked around it a bit when writing my own reactor.
> 
> -bob
> 
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 

-- 
Yun Mao
Ph.D Student, University of Pennsylvania
http://www.cis.upenn.edu/~maoy/






More information about the Twisted-Python mailing list