[Twisted-Python] newbie confusion - puzzling reactor response

K. Richard Pixley rich at noir.com
Fri Feb 12 19:51:29 MST 2010


Glyph Lefkowitz wrote:
> If you only have one disk, you can only really get a benefit from two, maybe three file I/O slave processes, and that's a fairly small amount of resources to manage.  Granted, it's tricky to really identify how many "disks" you've got in a system, and the performance characteristics change radically based on what kind of disk technology is involved, but generally speaking a few worker threads and a queue of I/O operations would cover the vast majority of use-cases.
I'm working with parallelized build servers.  We often have raided 
disks, solid state disks, servers with huge amounts of disk cache 
specifically so that an entire build happens in memory, etc.  File io is 
our bottleneck.

I think you probably are also forgetting about NFS.  NFS isn't slower 
than native disk in terms of throughput, only in terms of latency, which 
is a fabulous opportunity for asyncronous file io.  (Granted, NFS seems 
to have fallen out of fashion recently.)

I think twisted already has everything that's required.  It could 
probably use a slightly more friendly interface paradigm so the user 
doesn't have to do his own os.open, but really, even that wouldn't save 
much.

Reactor core was enough to sell me on twisted.  That's probably all I'll 
even be using.  And compared to writing my own, that's enough to be useful.

--rich




More information about the Twisted-Python mailing list