[Twisted-web] leightweight CMS

[James Y Knight]

On Mar 13, 2005, at 8:51 PM, Andrea Arcangeli wrote:
> In the long run twisted should support kernel async-io (and even more
> important epoll for different reasons), then you won't have to use an
> helper thread to do the bulk I/O. Using an sql server is the current
> best solution to keep everything async and it normally provides several
> other advantages too ;).

Of course you are correct, but I would disagree about the short-term 
importance. I suspect you'll get better performance actually by reading 
files blockingly than by using an SQL server. For a typical website, I 
expect all the most active content will fit in the cache quite handily, 
and that which is not as active can be read off disk quickly enough to 
not matter. If you read in small chunks, the disk will likely be able 
to read quicker than your kernel TCP buffers can become empty. However, 
notice my use of the "likely", "expect", "suspect" qualifiers: I have 
not actually *done* this measurement, but from my experience with SQL 
servers, they are not excessively speedy.

Spawning threads to do your reads is of course an option, but not one I 
would recommend worrying about unless you've actually measured it being 
a problem. And, if you *have* done that measurement, please help to 
write a generic async-disk-io module for Twisted which can use either 
threads or kernel async IO, as appropriate, and put this to rest once 
and for all. :)

James