[Twisted-Python] logging

Christopher Armstrong radix at twistedmatrix.com
Thu Jun 13 19:02:16 MDT 2013

On Thu, Jun 13, 2013 at 5:14 PM, Glyph <glyph at twistedmatrix.com> wrote:
> Hello everybody. Today I'd like to talk about everyone's favorite subsystem
> within Twisted: logging.
> There are a number of problems with Twisted's logging system.

I really need to get out of here and go to dinner, but I really
quickly wanted to start a list of use cases for logging.

1. "I need to convey a message with complex information to the user of
my software"
Log files usually aren't good at all for this, but they're
unfortunately used quite often for it. This is pretty much the only
reason to have English in your log statements, I think.

2. "I want to be able to analyze patterns in the execution of this software"
This is where structured data really helps. Keys, values, "systems",
and "events" (as I described below) -- things you can filter by,
graph, or redirect.

3. "I want to investigate certain events that happened in the
execution of this software"
Structured data helps just as much here as with #2. It's nice to be
able to interactively query databases of logs. "show me all the stuff
about this player". "show me everything that happened in this thread".
"show me all of the 'frag' events."

4. "I want to accumulate a bunch of extremely verbose data to be
logged only if it's really necessary, such as in an error case"
This is similar to #3, but there are cases where you have such verbose
data that you don't want to to log it all the time. This is basically
the OOPS system as some developers are familiar with -- the execution
of request-handling can attach data to the request (or some log
context object, or whatever) and if the request ever fails, all of
that data will be zipped up and saved somewhere for further
investigation. If the request succeeds, it'll be thrown out.

5. "I want to be able to track down where this log message came from"
the "event" attribute that I described is helpful here, but having the
filename and line number is even better.

I tried to write a number #6 about including framework-provided data
in your application-level log statements (like request ID, client IP,
URL etc), but I got too hungry.

Sorry for the braindump!

Christopher Armstrong

More information about the Twisted-Python mailing list