[Twisted-Python] Twisted, twistd and logging

[Tim Allen]

Phil Christensen <phil at bubblehouse.org> wrote:
> On Jun 19, 2008, at 2:21 AM, Tim Allen wrote:
> > My second thought was to use twisted.python.log, but that brings its
> > own set of problems:
> >
> > 1. twisted.python.syslog isn't very... complete. For starters, it
> >    doesn't let you specify the syslog facility, and even if it did,
> >    twistd would need to be extended to support it.
> 
> It's true that the syslog support is very experimental at this point,  
> but what would need to be changed about twistd to support the  
> necessary development?

As I understood it, twistd's philosophy was that logging-configuration
was a deployment issue and hence ought to be configured by tweaking
twistd command-line parameters rather than pre-configuring the
application to be deployed. twistd already has very simple logging
configuration options in the form of "--logfile" and "--syslog"; I
intended to add a "--syslog-facility=" command-line option [0].

Having skim-read ticket 638 that JP pointed to in another reply,
now I'm not so sure. It's kind of difficult to for me to understand the
current state of play, especially since that ticket predates the merge
of seemingly-related ticket 3052, but it *seems* to be a battle between
the camps of "twistd's logging is limited, it should be infinitely
extensible" and "twistd's logging is limited, applications
should be allowed to override it". Personally, I think I lean
more toward "it should be infinitely extensible" side - but right now I
don't really care about the general case of extensibility, I just want
more flexible syslog support.

At any rate, ticket 638 and other related tickets seem to be high-level
conceptual arguments, and I don't feel I have the domain-knowledge
required to comment intelligibly on them.

> I think the most likely scenario is that the standard logging code  
> assumes logging will return immediately, but I feel like this could be  
> worked around by having some kind of log queue that gets sent to  
> syslog by some background thread (or better yet, a callLater pattern).

I'm not sure what you're getting at here. Python's syslog module calls
syslog(3), which talks to syslogd(8). I don't know whether either or
any of those block waiting for the log message to be delivered; I would
hazard a guess that they don't.

> > I think the best way of solving my problems would be to:
> >
> > a) update t.p.syslog so that it supports a 'facility' parameter.
> > b) update t.p.syslog so that log-messages with 'isError' true are
> >    logged with the LOG_ALERT priority rather than the default.
> > c) update the Unix version of twistd to support a 'syslogFacility'
> >    command-line parameter.
> >
> > Would such a patch be accepted? If necessary, I could replace (b) with
> > a syslog-based LogObserver in my own code, but I'm keen to have (a)  
> > and
> > (c) in the Twisted code-base.
> 
> I can't speak to whether such a patch would be accepted, but I know I  
> have a couple of projects I've wanted to use syslogging for, and it  
> would be great to have a stable solution in Twisted. However, the core  
> dev team is fairly well inundated, so while you're working on a patch/ 
> review, you might consider investigating another option in parallel.
> 
> If you package your code separately and join the Twisted Community  
> Code project on Launchpad, you can get your code out there and in  
> front of eyeballs while you work on getting a patch accepted into  
> Twisted. If you can draw a reasonable amount of user attention, this  
> may help with both testing and review, and make it easier to get your  
> patch into Twisted proper.

Thanks for the advice - I'm not sure that patches to twistd could be
packaged separately, but I'll keep the possibility in mind in case I
think of anything.

Tim Allen

[0] I'm not familiar with t.p.usage; if it were possible to support
syntax like "--syslog=LOG_LOCAL0" while still supporting the "--syslog"
syntax in existing releases, that'd be ideal.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20080620/8b293bdb/attachment.pgp