[Twisted-Python] Twisted scalability question
matti.jagula at proekspert.ee
Mon Sep 22 16:58:55 EDT 2003
I've got a task to write a DHCP server for a local cable operator and I'm
considering different frameworks to write it in. I've looked at Twisted a
couple of times and thought I'd do some little experimenting to see how well
it behaves with the current task at hand.
The DHCP server needs to do some database interaction to support the business
logic. So to model the problem I wrote two simple QOTD servers for UDP
protocol, using a postresql backend for quotes.
One server is written using a Twisted DatagramProtocol and uses Deferreds for
database query using adbapi. The other is a dead-simple single-threaded
block-and-wait UDP socket server with a single database connection that just
reads from the udp socket, queries the database and writes the result back.
So far so good. I wanted to see how well these servers scale with a lot of
requests. First I tried making just a lot of sequential requests from a
single-threaded client, but that showed that the Twisted version was slower
by a factor of ~4.
After a little consideration I thought that Twisted must be better for a lot
of concurrent requests, because of the threaded database access. Twisted
should be able to accept more than one client at a time and keep all the
database connections (5 by default, i guess) busy. So I forked 20 clients off
using a simple bash script with a for loop. The result was that the Twisted
is still slower, by the same factor of 4.
The "top" shows that while the Twisted server is running, the 5 postgresql
connections are mostly in "idle" state (they change the name of the process
to show it). However, while the single threaded server is running, the only
active pgsql connection is mostly in its "select" state.
So, after the rather lengthy introduction, I'd like to know what am I doing
wrong? How can I show the better scalability of Twisted platform?
More information about the Twisted-Python