[Twisted-Python] Async file I/O with Linux & Twisted

[Michał Pasternak]

On Thu, 20 Dec 2007 09:23:43 +0000
"Arnar Birgisson" <arnarbi at gmail.com> wrote:

> Hi there,
> 
> May I ask why you chose libaio over, for example, libevent?

libevent is to C as Twisted is to Python. Both run on many
OS/platforms, but neither give you asynchronous file read/write;
read(2) on a file descriptor (not socket) will be always blocking.
At least this is what I understood after quick glance at libevent
source - all it does in buffer.c:evbuffer_read on non-win32 OS is
calling read(2).

libaio, instead, gives you async filesystem read/write access. Reading
a few hundred megabytes of data won't block Twisted reactor, which is
good.

Even better news is that since sending that e-mail yesterday I've found
out how to integrate Linux async I/O with epoll notification, so the
implementation I'm working on now will be even better. Pity it's
Linux-only, but well... to be portable, it should be POSIX aio really,
but I can't give you any guarantees about POSIX aio performance on
Linux. I can try working on other OSs later, if there will be enough
ppl interested.

Of course it would be cool to have Python libevent bindings - and even
cooler it would be to have a portable libevent reactor in C - but would
that really remove some of the limitations that Twisted has still to
deal with? You'd still have to hack portable AIO implementation into
libevent to have async file read/writes then ... :)

Take care,
-- 
m