<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Feb 12, 2010, at 9:51 PM, K. Richard Pixley wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>I'm working with parallelized build servers. We often have raided <br>disks, solid state disks, servers with huge amounts of disk cache <br>specifically so that an entire build happens in memory, etc. File io is <br>our bottleneck.<font class="Apple-style-span" color="#000000"><font class="Apple-style-span" color="#144FAE"><br></font></font></div></blockquote><div><br></div><div>Yeah, this level of disk manipulation is past the point where a little bit less blocking on I/O will help... I imagine you've already got some kind of process/thread pooling solution already, or at least you'll need one.</div><br><blockquote type="cite"><div>I think you probably are also forgetting about NFS. NFS isn't slower <br>than native disk in terms of throughput, only in terms of latency, which <br>is a fabulous opportunity for asyncronous file io. (Granted, NFS seems <br>to have fallen out of fashion recently.)<br></div></blockquote><div><br></div><div>No, I'm not forgetting about it: I'm just saying that once you've got an API that applications can start using that gives *some* performance benefit (non-blocking disk I/O at the expense of spinning up a few threads / processes behind the scenes), you can always optimize it for other use-cases later, without necessarily changing the API.</div><br><blockquote type="cite"><div>I think twisted already has everything that's required. It could <br>probably use a slightly more friendly interface paradigm so the user <br>doesn't have to do his own os.open, but really, even that wouldn't save <br>much.<br></div></blockquote><div><br></div><div>It would allow us to do it more portably, I think. os.open()'s behavior can vary a lot depending on what you do with it.</div><br><blockquote type="cite"><div>Reactor core was enough to sell me on twisted. That's probably all I'll <br>even be using. And compared to writing my own, that's enough to be useful.<br></div></blockquote></div><br><div>Great, glad to hear it!</div></body></html>