[Twisted-Python] logging

[Christopher Armstrong]

On Thu, Jun 13, 2013 at 5:14 PM, Glyph <glyph at twistedmatrix.com> wrote:
>
> Hello everybody. Today I'd like to talk about everyone's favorite subsystem within Twisted: logging.
>
> There are a number of problems with Twisted's logging system.

Thanks for writing up this email. I'd like to help out with this effort.

> For a long time, there's been consensus among at least a few core Twisted developers that log messages ought to be specified in terms of a format message and several relevant keyword arguments, so that tools can extract the values of the keyword arguments without parsing ad-hoc messages or using regexes. In other words, like this:
>
> log.msg(format="Foo happened. Relevant information includes: "
>                 "bar=%(bar)s baz=%(baz)s boz=%(boz)s",
>                 bar=bar, baz=baz, boz=boz)

There's something that I want to.... not propose, but discuss, that
I've discovered through my writing of actual applications that do
actual log filtering.

Basically, the conclusion I've come to is that English isn't very
useful in logs. It's usually badly written, uninformative, and hard to
filter on. The main thing that I find really important is having what
I call an "event name", which has the following properties:

1. human readable
2. computer readable
3. mandatory
4. unique across a codebase, and specific to the physical logging
statement that it's being passed to.

So, for example:

def frag_message_received(self, fragger, fragee, weapon):
    self.log("frag", fragger=fragger, fraggee=fragee, weapon=weapon)
    ...

Now, your smart log filterer can just filter by event="frag" instead
of message="Frag for \w+ received with weapon \w+" or whatever.

This is just how I've done things in practice, and it has worked very
well for fairly large projects, but there are improvements that I can
think of. For example, the event name should probably be based on a
hierarchy of composable logger objects (kinda like the one David Reid
wrote) so that it's easier to maintain uniqueness in a large codebase.

(Hey David, you should write an email about that logging class :)

--
Christopher Armstrong
http://radix.twistedmatrix.com/
http://planet-if.com/