<div class="gmail_quote">On Wed, Aug 26, 2009 at 4:56 PM, Martin-Louis Bright <span dir="ltr"><<a href="mailto:mlbright@gmail.com">mlbright@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<p>I would like to write a small daemon that monitors (tails) a
server log, parses the entries and sends HTTP requests based on some of those
entries. I would like it if the reading of the log file and the
sending of http requests were asynchronous. Should I use twisted for this? Or is twisted overkill...</p></blockquote><div>Twisted would be perfectly appropriate for this! The HTTP requests will certainly be non-blocking, assuming you use twisted.web.client (or the new, better thing which I hope will be released before we all get old).</div>
<div><br></div><div>In fact, if anything, Twisted is under-kill; you'll need to go a bit further. The one minor issue is that Twisted has no explicit way to do asynchronous file I/O (because most operating systems provide a bewildering array of not-really-working systems for non-blocking file I/O, so it would be challenging for Twisted to do in a way that was useful). There are a number of ways to emulate it though; for log files (which are fairly low volume, and almost by definition will immediately be in your filesystem cache as the parts you want to read become available) you can just do a LoopingCall() which does .read() on a file that it leaves open to retrieve the next chunk of data, and that will probably be good enough. In practice you won't actually block doing this read() because your kernel is just going to pull the bytes directly out of filesystem cache memory and hand them back to you. It is of course possible to have to wait for the disk or even the network, depending on your underlying filesystem.</div>
<div><br></div><div>I just filed <a href="http://twistedmatrix.com/trac/ticket/3983">http://twistedmatrix.com/trac/ticket/3983</a> for a more thorough solution (mostly in the hopes that somebody will close it as a duplicate and point to some pre-existing issue I couldn't find through search) so you can monitor future discussion there if you like.</div>
</div>