[Twisted-Python] deciding to use twisted or not

[Cary Hull]

On Wed, Aug 26, 2009 at 6:41 PM, Glyph Lefkowitz <glyph at twistedmatrix.com>wrote:

> On Wed, Aug 26, 2009 at 4:56 PM, Martin-Louis Bright <mlbright at gmail.com>wrote:
>
>> I would like to write a small daemon that monitors (tails) a server log,
>> parses the entries and sends HTTP requests based on some of those entries. I
>> would like it if the reading of the log file and the sending of http
>> requests were asynchronous. Should I use twisted for this? Or is twisted
>> overkill...
>>
> Twisted would be perfectly appropriate for this!  The HTTP requests will
> certainly be non-blocking, assuming you use twisted.web.client (or the new,
> better thing which I hope will be released before we all get old).
>
> In fact, if anything, Twisted is under-kill; you'll need to go a bit
> further.  The one minor issue is that Twisted has no explicit way to do
> asynchronous file I/O (because most operating systems provide a bewildering
> array of not-really-working systems for non-blocking file I/O, so it would
> be challenging for Twisted to do in a way that was useful).  There are a
> number of ways to emulate it though; for log files (which are fairly low
> volume, and almost by definition will immediately be in your filesystem
> cache as the parts you want to read become available) you can just do a
> LoopingCall() which does .read() on a file that it leaves open to retrieve
> the next chunk of data, and that will probably be good enough.  In practice
> you won't actually block doing this read() because your kernel is just going
> to pull the bytes directly out of filesystem cache memory and hand them back
> to you.  It is of course possible to have to wait for the disk or even the
> network, depending on your underlying filesystem.
>
> I just filed http://twistedmatrix.com/trac/ticket/3983 for a more thorough
> solution (mostly in the hopes that somebody will close it as a duplicate and
> point to some pre-existing issue I couldn't find through search) so you can
> monitor future discussion there if you like.
>

It would certainly be nice if Twisted supported async file io, but in this
case wouldn't a ProcessProtocol around 'tail -f' be a good solution as well?

-Cary


>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>


-- 
01100011 01100001 01110010 01111001
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20090826/f9082060/attachment.html>