[Twisted-Python] logging

Glyph glyph at twistedmatrix.com
Sat Jun 15 04:19:14 MDT 2013

On Jun 14, 2013, at 10:43 PM, Christopher Armstrong <radix at twistedmatrix.com> wrote:

> I don't think it's worth coming up with some kind of GUID-based
> system, because I don't think anyone's going to go to the trouble to
> use it, and I think it basically offers no practical benefit over
> simple event names.

Here are the benefits over simple event names:

It's opaque, so as code moves around, it doesn't look dated or wrong.  Your 'system' argument looks a whole lot like a FQPN, but as you keep saying, code moves around a lot.  Having a *second* dotted identifier that looks a lot like the module name is going to look a lot like cruft to someone when it stops matching a module name, they're going to be motivated to "clean it up", and then the "cleaned up" version is going to break everyone's logging instrumentation.
Since UUIDs can be derived from other information, you can start off without one and add one later.  If you didn't feel like specifying one in the first place with your log messages, then you're stuck until the monitoring and application code that comes to some agreement.

My view of our present disagreement seems to come down to this: I believe that nobody is going to bother to do logging right in the first place (particularly, they are not going to take the time to specify or enumerate the types of events that are interesting that their system emits), and will only realize they need to extract interesting stuff from their logs later.  I think we need to provide the best possible support for the developer "later" (the opaque UUID they can associate with an API symbol) and the monitoring folks "now" (the derived UUID that they can use in place of a gross and broken-in-the-future regex).

This doesn't mean that the UUID magically makes monitoring always work in the future; developers can still change their format strings willy-nilly.  But, it at least provides a mechanism that they *could* use to avoid breakage if everyone believes it should be avoided.

I understand your point to be that you think that people should, and that they therefore will, go to the trouble to categorize every thing that gets logged as they're writing the logging stuff.

In support of my argument, I offer as evidence the unfortunate mess that is Twisted's current ad-hoc usage of logging, plus the existence of Splunk, a publicly-traded company whose whole reason for existing is that they can run regexes over poorly-structured logs to extract useful information and statistics from them :-).

To be fair, one element in support of your argument is that you managed to write a system that used this idiom and it worked well in practice.  I'm sure that it did, and I think it's a good thing for people to adopt.

My concern is that lots of useful log-worthy events are coming out of an open-source library that won't be exposed to the discipline that a particular team adopts for logging.

Finally, it's worth noting that GUID-based identifiers and textual, hierarchical identifiers are not mutually exclusive.  You can have log.info(log_guid="98185C94-57D7-439A-8206-1D4A2ACBD983", log_system="spaceships.combat.explosion") in a single event.  We could provide both tools to the developer, and they're not obliged to use either.

(I see you've written some other messages, so more forthcoming...)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130615/d2b1cfc8/attachment.html>

More information about the Twisted-Python mailing list