[Twisted-Python] twisted socksv4 proxy

[Bob Ippolito]

You won't be able to register a callback to it.. on second thought, you 
may be able to proxy writeSomeData and look at the result to see how 
much data got written, but I think that the reactor may have the actual 
FileDescriptor registered so you'd have to do some more trickery.

On Wednesday, Sep 24, 2003, at 01:16 America/New_York, Yun Mao wrote:

> Is it true that the only way to see the success of write() is to check 
> the
> FileDescriptor.dataBuffer?
>
> I'm thinking that suppose there is one way to check this, and I can
> somehow add a callback function to the success of each write(), then I 
> can
> simply call source.stopReading() after write() and startReading() when 
> I
> get callback.
>
> Yun
>
>
> On Wed, 24 Sep 2003, Bob Ippolito wrote:
>
>> On Wednesday, Sep 24, 2003, at 00:08 America/New_York, Yun Mao wrote:
>>
>>> I tried to use twisted python to write a proxy following the given
>>> socksv4 example.
>>>
>>> The problem i discovered is that: when i try to forward two
>>> connections,
>>> their capacities may not match.
>>>
>>> In a blocking implementation,
>>>            data = source.read(); dest.write(data);
>>>
>>> In a non-blocking implementation, what socksv4 did is something like:
>>>
>>>    def dataReceived(data):
>>> 	otherConn.transport.write(data)
>>>
>>> However, if source is a much faster link than dest, would there be a
>>> problem? write always return even it should block so that read always
>>> happens, and the buffer of source tcp connection will never get full,
>>> to
>>> let tcp start flow control?
>>
>> With that code, you are right.  Twisted will buffer everything for
>> dest.. so you won't lose any packets, but you might start using a lot
>> of memory and source may think that dest has a faster connection than
>> dest actually does.
>>
>> The twisted.internet.policies module can throttle transports, however 
>> I
>> don't think that any of them can actually determine how fast dest is
>> actually consuming the data.  That would require introspecting the
>> underlying transport's buffer and I don't believe the transport
>> interface has any way of doing that.
>>
>> You could break the rules and assume that there is a dataBuffer
>> attribute (from twisted.internet.abstract.FileDescriptor) which would
>> most likely work, until you try and use it on a transport that is not 
>> a
>> FileDescriptor (wrapped by another policy, is a loopback for testing,
>> using a custom reactor, etc.).
>>
>> In any case, yes, it's possible to solve this problem, but you will
>> have to write your own throttling policy which must bend the interface
>> rules in its implementation.
>>
>> I would suggest writing the throttling policy code, and then posting
>> your code with a description of your use case to
>> http://twistedmatrix.com/bugs/ so there can be a discussion about how
>> to best change Twisted's interfaces around a bit so that this kind of
>> throttling policy can be implemented in a future release of Twisted
>> without digging into objects beyond their declared interfaces.
>>
>> I could be mistaken, though.  I'm not an expert on twisted.internet,
>> but I poked around it a bit when writing my own reactor.
>>
>> -bob
>>
>>
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>
> -- 
> Yun Mao
> Ph.D Student, University of Pennsylvania
> http://www.cis.upenn.edu/~maoy/
>
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python