heya,<div><br></div><div>Thanks to everybody for their replies =).</div><div><br></div><div>@Jason Rennie: Thanks for the links - I suspect there's going to be a bit of a curve to me getting on board with Twisted =).</div>
<div><br></div><div>Still not sure if Twisted will satisfy all of my requirements, but I'll need to do some more investigation.</div><div><br></div><div>However, what did you mean by it may be "nontrivial to use twisted with non-twisted friendly libraries"? What would make a library non-Twisted friendly?</div>
<div><br></div><div>Not sure what libraries we'll be depending on, but watchdog (<a href="http://packages.python.org/watchdog/installation.html" target="_blank">http://packages.python.org/watchdog/installation.html</a>) might be just one example.</div>
<div><br></div><div><div>@Glyph Lefkowitz: Aha, thanks for the headsup about Solaris. I probably should have checked that myself...lol.</div><div><br></div><div>In terms of the buildbot, I'm not sure I'm the most suitable person. The primary reason I use Solaris is because my workplace is partly a Solaris shop - and even then, I have minimal contact with most sysadmin aspects of it.</div>
</div><div><br></div><div>And my experience with OpenCSW and the awesome team there shows that some of the quirks on the Solaris platform (particularly the older versions) can be a pain to work around, and getting stuff to build cleanly on it can often be non-trivial, at least if you're not experienced with the platform.</div>
<div><br></div><div>@Johan Rydberg: Reliable delivery is definitely an issue - one of the production management guys today said "whatever you're hacking up, we want to make sure we don't miss any lines."</div>
<div><br>
</div><div>The Scribe logging server you (<a href="https://github.com/facebook/scribe/wiki" style="color:rgb(0, 109, 175) !important;outline-style:none;outline-width:initial;outline-color:initial;font-family:Arial, Helvetica, FreeSans, sans-serif;font-size:13px;line-height:17px;text-align:left;background-color:rgb(255, 255, 255)" target="_blank">https://github.com/facebook/scribe/wiki</a>) mentioned actually looks very interesting. </div>
<div><br></div><div>Another avenue I'm looking at the moment is RabbitMQ - the idea being I hook up a pseudo "tail -f" in Python, and use that to publish </div><div>to a RabbitMQ instance. (I have little experience with messaging buses before this - it's only something I'm learning about now, to be honest).</div>
<div><br></div><div>Then, I just get each application that needs to monitor log events to just subscribe to the appropriate queue. The thing I like about this approach is that it decouples the producers/consumers somewhat - we can have multiple different applications spread across several monitoring boxes subscribing to the same events. That, and I suspect it might scale a little bit better than any slipshod job with Twisted I'm able to hack together in the time I have. Thoughts?</div>
<div><br></div><div>Cheers,</div><div>Victor</div><div><br></div><div><div class="gmail_quote">On Tue, Sep 27, 2011 at 14:09, Victor Hooi <span dir="ltr"><<a href="mailto:victorhooi@yahoo.com" target="_blank">victorhooi@yahoo.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<div><br></div><div>We have several Solaris 10 running an application that generates large text logfiles</div><div>
<br></div><div>We need to stream these logfiles in realtime to a central monitoring server.</div><div><br>
</div><div>Unfortunately (for both technical and non-technical reasons), we can't use an NFS-mount. At the moment, we're using SSH with tail -f to stream the files from the servers to the monitoring box.</div><div>
<br></div><div>However, I was going to look at hacking a client/server up in Python to monitor for filesystem events, and then stream any updates.</div><div><br></div><div>In terms of filesystem monitoring, this is Solaris 10, so there's not really that much available it seems. Gamin seems to be the best bet, otherwise I might have to fallback to polling.</div>
<div><br></div><div>On the Twisted side, is the above something that's fairly easy to accomplish? I haven't used Twisted before, so I thought this might be a good excuse to learn it. Any thoughts or caveats I should be aware of? Any existing projects I might be able to leverage off? Good starting docs for this sort of thing?</div>
<div><br></div><div>Thanks,</div><div>Victor</div>
</blockquote></div><br></div>