[Twisted-Python] Persuading Python's Logging to use twisted.logger
Glyph
glyph at twistedmatrix.com
Mon May 9 16:16:17 MDT 2016
> On May 9, 2016, at 1:50 PM, Daniel Sutcliffe <dansut at gmail.com> wrote:
>
> The project I am working on uses pymodbus which I am sure shares a
> fairly common attribute with many other modules of using Python's
> standard Logging mechanism - a very reasonable choice even for a
> module that supports Twisted, the library can also be used entirely
> synchronously and thus would not want a required dependency of
> Twisted.
Sure, this makes sense.
> It struck me that it would be great to be able to redirect the
> standard logging library to use twisted.logger by some sort of 'Clever
> Monkey Patching' and that this may be a relatively common
> requirement... however after extensive searching, and asking on the
> pymodbus list, I can't find any evidence that such a thing has ever
> been attempted or discussed.
You shouldn't need monkey patching; the standard library logger does have facilities for setting up custom handlers. We just need to make one. Honestly it would be great if this were featured within Twisted itself; it would make a great contribution to twisted.logger proper.
> The reverse mechanism of sending twisted.logger's output to the
> standard library is of course handled by the
> twisted.logger.STDLibLogObserver (and similar in twisted legacy
> logging) but the documentation for this even suggests why this is a
> bad idea: 'Warning: specific logging configurations (example: network)
> can lead to this observer blocking.' which seems to me why it would be
> better to attempt this the other way around...
I would very much like to see this :). The standard library logging mechanism is in common use, and until we can spin it out into a separate project, `twisted.logger´ is unlikely to
> Am I crazy to even think this? is it just the rambling of
> Python/Twisted newb? Or is there something I'm missing that would make
> this impossible to do generically, and awkward to provide a vague
> recipe of how to do?
No, there's nothing that would make it impossible. If you want to contribute it.
> I do appreciate that twisted.logger offers a more feature rich
> (structured) API and the Logging API would only be able to provide
> level and text but it would be better than loosing any possibly useful
> log messages from used modules in my mind.
I think you might even be able to claw back some structured information out of the stdlib logging machinery? LogRecord does have 'args' and 'msg', and we could convert that into a twisted.logger format string. This would be more expensive than using twisted.logger directly of course, but it would still provide a lot more useful information.
> If anyone can enlighten me I would be most appreciative,
I think just try to write a Handler (<https://docs.python.org/2.7/library/logging.handlers.html#module-logging.handlers>) that converts LogRecord objects to event dictionaries, and then emit them to a <https://twistedmatrix.com/documents/16.1.1/api/twisted.logger.ILogObserver.html> and let us know if you hit any problems.
-glyph
More information about the Twisted-Python
mailing list