[Twisted-Python] Twisted scalability question

[Matti Jagula]

Hi,

I've got a task to write a DHCP server for a local cable operator and I'm 
considering different frameworks to write it in. I've looked at Twisted a 
couple of times and thought I'd do some little experimenting to see how well 
it behaves with the current task at hand.

The DHCP server needs to do some database interaction to support the business 
logic. So to model the problem I wrote two simple QOTD servers for UDP 
protocol, using a postresql backend for quotes. 

One server is written using a Twisted DatagramProtocol and uses Deferreds for 
database query using adbapi. The other is a dead-simple single-threaded 
block-and-wait UDP socket server with a single database connection that just 
reads from the udp socket, queries the database and writes the result back.

So far so good. I wanted to see how well these servers scale with a lot of 
requests. First I tried making just a lot of sequential requests from a 
single-threaded client, but that showed that the Twisted version was slower 
by a factor of ~4. 

After a little consideration I thought that Twisted must be better for a lot 
of concurrent requests, because of the threaded database access. Twisted 
should be able to accept more than one client at a time and keep all the 
database connections (5 by default, i guess) busy. So I forked 20 clients off 
using a simple bash script with a for loop. The result was that the Twisted 
is still slower, by the same factor of 4.

The "top" shows that while the Twisted server is running, the 5 postgresql 
connections are mostly in "idle" state (they change the name of the process 
to show it). However, while the single threaded server is running, the only 
active pgsql connection is mostly in its "select" state.

So, after the rather lengthy introduction, I'd like to know what am I doing 
wrong? How can I show the better scalability of Twisted platform?

-- 
  Matti Jagula