[Twisted-Python] Guidance on Proxy-type Application

[John Santos]

Hi Adam -

I'm in pretty much the same situation as you and am working on a similar
problem.  Very much a Python and Twisted newbie.

The program I'm working on needs to talk to a variety of clients and to
a single server, with more-or-less permanent connections to all of them.

My problem is somewhat different from yours (and probably a little
simpler), since all input from all clients goes to the server, and
all output from the server goes to all clients (so I don't have to
tag and remember which response goes to which client.)  However,
there are other messages from the clients that don't go to the
server, but instead do other things, some requiring responses.
So the message handling in my program (what happens when dataRecieved
gets called) may be more complicated than in yours.  But that is
(at least from my perspective) the easy part.  Getting the twisted
app to talk to the server and all the clients, and not to fall over
dead when a client goes away, etc. is (to me) the hard part, and I
feel like I've just gotten over a huge hump now that that is (mostly)
working (as of two days ago.)


I am using two factories, one to generate the object that talks to the
server (using conch.telnet), and one to generate objects to talk to
the clients.  The clients can come and go but would typically be
permanent.  (Actually 3 factories right now, since I'm using the
StdioProxyFactory from the example for debugging, but that will
go away as I get the client protocol working.)

I started from the dataforward.py example in Twisted book, but by
now it has been massively changed.

Before starting the reactor, I do a reactor.connectTCP to the server
using the factory for the server connections and a reactor.listenTCP
using the client connection factory, to listen for client connections.

One change I had to make to the example was to promote the
InputForwarder object from an attribute of the client connection
object to a global object since it is common to all the
client connections.  (I hope I have the terminology correct here;
as I've said, I'm a Python newbie.)

It turns out to be fairly small amount of code, and I think it
can still be simplified some more.

So I think you are on the right track.  HTH!


-- 
John Santos
Evans Griffiths & Hart, Inc.
781-861-0670 ext 539
-------------- next part --------------
I have just started to look at the Twisted framework and would like to put

it to use for a new project I am working on.  Not being very familiar with

the framework and fairly new to Python in general I would like to ask a

design/architecture question.  (I have written similar applications in C but

would prefer to start this in the right direction and not write Python like

C.)



The application has the following model:



Many clients connect to the Application and prefer to leave the connection

open.  They will send messages across this connection.  They will expect to

get a message back at some point later, they do not wait for a response

(async).  The clients are already coded (legacy) and just need to send their

proprietary protocol to the new Application (written using Twisted).



The Twisted application will take the data from the clients and do some

transformation on it then send the message on to another server (3rd

party).  This connection to "another" server must be a single connection,

not one connection per client.  This connection should also be persistent

and not opened/closed for each client message sent.  Ideally if the 3rd

party server is down then I would also not accept client connections as the

messages are time sensitive and should not be stored and forwarded.  At some

point the 3rd part will send a message back and the Application will route

it back to the original source.  Basically request/reply pattern.



I have been reading through the archives and the twisted docs and have also

looked over the Hex-dump port-forwarding recipe but not found anything that

explains how to use twisted for this model.  Hex-dump is close but

opens/closes the connection to the server on each client connection.



I am thinking that there will be two Factories [and two protocols: 1) for

clients and 2) for 3rd party].  I am not sure how to best establish both the

listening factory and the client to 3rd party factory.  Once they are

established what is the preferred way in Twisted to pass a message from one

protocol to another?



Any pointers or sample code that you can offer is greatly appreciated.  I

would really like to cook this in Twisted and not go back to the C way.



Thanks,

-ab

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20090528/f89ace8e/attachment.htm