[Twisted-Python] Deffered question
matteo.giacomazzi at libero.it
Tue Oct 21 09:59:57 EDT 2003
> I need to implement a command based protocol, that is a client sends
> a command to the server and waits for the response. Since the
> response computation might be time consuming I have to choose
> between 2 solutions:
> - use Twisted deffereds
> - or write my own implementation, a TCP based threaded server. I'm
> not worried about scaling since it will not be used by more than 10
> clients at a time.
I'm facing a problem like this.
I used to have a multithreaded server (fully written using Python)
that accepted commands via a TCP or UNIX socket, elaborate the
response (tipically querying some remote servers via HTTP) and then
returned the response: one query, one thread.
This server runs since 2001, I updated it, upgraded and refactored
some parts, but it's still kinda "heavy" to manage.
So, I realized that multithred is not _always_ the solution.
In my case, the server was running (with a web server and a DB server
too) on a dual Xeon architecture. So, at most, two threads are running
on the same time but there are more chances that only one of them is
running while the other CPU is used by the web server or the DB.
That's the solution I'm building:
- there are different _processes_, one for every kind of remote server
I have to query;
- _processes_ may run on different (hardware) servers;
- processes communicate via a "message passing process" I wrote with
So: one process accepts commands via TCP or UNIX socket, it quickly
parses the command and sends a message to the right ``server''; this
``server'' process the command, does-whatever-it-needs and then send
the response to the first process that, in turn, quickly translates it
in a ``line oriented'' way and returns to the caller.
All the servers are single-thread: that's because I need only to call
a remote host and to compute very very simple operations. Maybe in
your case threads would help - but how many threads?
I would be interested in hearing opinion about this architecture from
More information about the Twisted-Python