[Twisted-Python] newbie confusion - puzzling reactor response
K. Richard Pixley
rich at noir.com
Fri Feb 12 19:51:29 MST 2010
Glyph Lefkowitz wrote:
> If you only have one disk, you can only really get a benefit from two, maybe three file I/O slave processes, and that's a fairly small amount of resources to manage. Granted, it's tricky to really identify how many "disks" you've got in a system, and the performance characteristics change radically based on what kind of disk technology is involved, but generally speaking a few worker threads and a queue of I/O operations would cover the vast majority of use-cases.
I'm working with parallelized build servers. We often have raided
disks, solid state disks, servers with huge amounts of disk cache
specifically so that an entire build happens in memory, etc. File io is
our bottleneck.
I think you probably are also forgetting about NFS. NFS isn't slower
than native disk in terms of throughput, only in terms of latency, which
is a fabulous opportunity for asyncronous file io. (Granted, NFS seems
to have fallen out of fashion recently.)
I think twisted already has everything that's required. It could
probably use a slightly more friendly interface paradigm so the user
doesn't have to do his own os.open, but really, even that wouldn't save
much.
Reactor core was enough to sell me on twisted. That's probably all I'll
even be using. And compared to writing my own, that's enough to be useful.
--rich
More information about the Twisted-Python
mailing list