[Twisted-Python] Guidance on Proxy-type Application
Aaron Bush
asb.bush at gmail.com
Tue Jun 2 13:20:22 MDT 2009
glyph,
Sorry for the delayed response. I wanted to thank you for providing such a
complete example. This is excellent. Now I just need to wrap by head
around how it all goes together and works.
The logic of the queue append and pop of a deferred in ProxyClient
forwardLine and lineRecieve are throwing me for a loop. I'll keep tracing
it and get it down.
Thanks Again!
-ab
On Thu, May 28, 2009 at 6:58 PM, <glyph at divmod.com> wrote:
>
> On 01:23 am, asb.bush at gmail.com wrote:
>
>> I have just started to look at the Twisted framework and would like to put
>> it
>> to use for a new project I am working on. Not being very familiar with
>> the
>> framework and fairly new to Python in general I would like to ask a
>> design/architecture question. (I have written similar applications in C
>> but
>> would prefer to start this in the right direction and not write Python
>> like
>> C.)
>>
>
> Thanks for asking!
>
> I apologize for the delay in my answer. I started writing up a simple
> example (attached) but was discouraged to find that it was 100 lines long
> and required too much explaining.
>
> Then I started documenting it and explaining every line but that was a very
> long, tedious message. So, it doesn't have much in the way of explanation;
> I hope you will find it useful regardless.
>
>> The application has the following model:
>>
>
> Many clients connect to the Application and prefer to leave the connection
>> open. They will send messages across this connection. They will expect
>> to
>> get a message back at some point later, they do not wait for a response
>> (async). The clients are already coded (legacy) and just need to send
>> their
>> proprietary protocol to the new Application (written using Twisted).
>>
>
> This is *almost* a FAQ. At least, you may find this to be a useful answer:
>
> <
> http://twistedmatrix.com/trac/wiki/FrequentlyAskedQuestions#HowdoImakeinputononeconnectionresultinoutputonanother
> >
>
>> The Twisted application will take the data from the clients and do some
>> transformation on it then send the message on to another server (3rd
>> party).
>> This connection to "another" server must be a single connection, not one
>> connection per client. This connection should also be persistent and not
>> opened/closed for each client message sent. Ideally if the 3rd party
>> server
>> is down then I would also not accept client connections as the messages
>> are
>> time sensitive and should not be stored and forwarded. At some point the
>> 3rd
>> part will send a message back and the Application will route it back to
>> the
>> original source. Basically request/reply pattern.
>>
>
> The example that I've attached does basically this. Run it and then run
> 'telnet localhost 4322', and type some lines; you will see that they are
> transformed and echoed back to you, both by the proxy and by the protocol
> being proxied.
>
> At a high level, the answer to your question is so simple that it's hard to
> express. Basically, you just need to have all the relevant objects having
> references to each other, and calling methods to achieve the desired effect.
> The less magic, the better.
>
> More precisely, you need an object responsible for managing your outgoing
> connections to your legacy server, so that it can handle disconnection and
> reconnection, queueing messages and so on. Then you need your proxy server
> factory to hold a reference to that object, so that it can create references
> from each proxy server protocol connection object to the connection manager.
>
> This is related to another recent thread - you can see my message in that
> thread here:
>
> http://thread.gmane.org/gmane.comp.python.twisted/18377/focus=18385
>
>> I have been reading through the archives and the twisted docs and have
>> also
>> looked over the Hex-dump port-forwarding recipe but not found anything
>> that
>> explains how to use twisted for this model. Hex-dump is close but
>> opens/closes the connection to the server on each client connection.
>>
>
> I'm not sure why hex-dump port-forwarding is particularly relevant to this
> example. Is it just because this is an application that connects from one
> host to another?
>
>> I am thinking that there will be two Factories [and two protocols: 1) for
>> clients and 2) for 3rd party]. I am not sure how to best establish both
>> the
>> listening factory and the client to 3rd party factory. Once they are
>> established what is the preferred way in Twisted to pass a message from
>> one
>> protocol to another?
>>
>
> This part of your question is almost exactly the FAQ I mentioned above :).
> To reiterate that answer, you just need to have references between objects,
> and call methods on the objects you want to do stuff.
>
> If you have a client connection object, just get a reference to that from
> the relevant server connection object and call methods on the client object
> to emit messages on the client protocol, handling any responses
> appropriately. Deferreds can help with that latter part.
>
> It is always better if you can establish that reference as simply as
> possible; for example, by passing parameters to the __init__ of various
> classes. Again, for reasons that have nothing to do with Twisted
> specifically, it's a bad idea to try to establish these references by having
> global variables floating around.
>
> Here's a very very simple example of the "good way" to propagate some data
> to protocol instances that need it:
>
> class MyProtocol(Protocol):
> def __init__(self, data):
> self.data = data
>
> class MyFactory(Factory):
> def __init__(self, data):
> self.data = data
>
> def buildProtocol(self, addr):
> return MyProtocol(self.data)
>
> reactor.listenTCP(8765, MyFactory("some data"))
>
> and here's a simple example of a really bad way (don't do this!):
>
> class MyProtocol(Protocol):
> def connectionMade(self):
> self.bleh = bleh
>
> bleh = "some data"
> f = Factory()
> f.protocol = MyProtocol
> reactor.listenTCP(9876, f)
>
> Even in C, I'm pretty sure it's better style to pass structures to
> functions than to abuse piles of local variables :). I only illustrate this
> bad style here because it seems to be a common antipattern. The Protocol
> class itself doesn't take any parameters to __init__, and Twisted's users
> don't always realize that protocols and factories and so on are just regular
> objects, with no special rules; they just get methods called on them by the
> reactor.
>
>> Any pointers or sample code that you can offer is greatly appreciated. I
>> would really like to cook this in Twisted and not go back to the C way.
>>
>
> Based on what you've said so far, I think you're basically on the right
> track. Good luck!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20090602/ef3801aa/attachment.html>
More information about the Twisted-Python
mailing list