[UPDATE] twisted-linux-aio now works with epoll [was: Re: [Twisted-Python] Async file I/O with Linux & Twisted]

[Michał Pasternak]

On Thu, 20 Dec 2007 21:13:26 +0000
"Arnar Birgisson" <arnarbi at gmail.com> wrote:

> Kernel level AIO, is that about the kernel providing that abstraction
> or is it using the hardware in some entirely different way?

Linux KAIO (Kernel Async I/O) layer allows you to read and write
files in an async (non-blocking) way, just like you read or write
non-blocking network sockets. It is similar to async I/O defined by
Open Group:

	http://www.opengroup.org/onlinepubs/009695399/functions/aio_read.html

It is a quite new topic, because until now, you were able to perform
async operations only on network sockets. Async I/O in terms of
network sockets is nothing new. FreeBSD's kqueue, Linux's epoll are
just a means of notification, that a given async socket has some data
to read (or accepts data to be written), so you don't need to poll for
those events (like traditional Unix's poll, which is slow -- see
http://www.kegel.com/c10k.html ).

[UPDATE] twisted-linux-aio now works with epoll.
I finally found out how to create fd notification for aio, so right now
twisted-linux-aio supports it. It integrates well with epollreactor. I
also added some code to implement Python queue in it, so the usage
ATM looks like this:

  aio.Queue().readfile("/some/file/name").addCallbacks(...)

It is still work in progress, though. Too early to say about
efficiency, but it seems, that Twisted applications which do a lot of
concurrent disk I/O will benefit. Get the new code via
http://twisted-linux-aio.googlecode.com/ .

-- 
Michal Pasternak