[Twisted-Python] stdlib logger, loggerFor and filtering events

[Glyph]

> On Jan 5, 2018, at 7:11 PM, Felipe Dau <dau at riseup.net> wrote:
> 
> As the logger has been brought up, I'd like to ask a couple things:


Just for the record, you don't need to wait for it to be brought up before asking other questions ;-).

> 1) When I was working on adding logs to an application, I wanted it to
> use both the `textFileLogObserver` and the `STDLibLogObserver`. As the
> application could be used either with its CLI or GUI, it seemed
> interesting to have both these loggers available. I was able to do
> that but I wanted them to have the same format - specifically, I
> wanted to use the file's format (which is awesome btw) - and had to
> override some parts of the `STDLibLogObserver`. It would have made
> things a lot simpler if that observer accepted a function to format
> events like the `FileLogObserver`.

It might be cool to add that, but, if you don't want the standard library's log format, why are you using the standard library log observer?  And wouldn't this be a job for a https://docs.python.org/3.6/library/logging.html#logging.Formatter <https://docs.python.org/3.6/library/logging.html#logging.Formatter> object in any case; what Twisted should be doing is sending the logs on in a more structured format that works with that?

> 2) Why is `_loggerFor` "private/hidden"? That is a great factory which
> imo should have more visibility - and also works great with attrs!:
> 
>     attr.ib(default=attr.Factory(_loggerFor, takes_self=True))

I'm honestly not sure why this exists at all.  It seems like a terrible mistake has happened here, since `_loggerFor` is actually exported in __all__, which should never ever happen.

From what I can tell, it should be removed, since rather than creating that fairly noisy attr.ib() definition, you could do this:

    _log = Logger()

which is a lot more succinct, and has exactly the same effect (do `self._log.info <http://log.info/>(...)`) and there you have it.

If you could explain why it's great, then maybe we could remove the underscore and add it to the docs.

> 3) Is there a way to suppress logs from libs or any other code that
> is not your own? In my case, I had to manually set which namespaces
> I wanted to filter but, for example, when you subclass a third-party's
> class which logs something, that will pass because now it belongs to
> one of your namespaces. I know it sounds crazy but wanted to know what
> other people think.

Absolutely!  Super glad you brought this up. This was a major part of the design of the new logging system, both to do this, and to also do as little work as possible with messages that were ultimately caught by such filters.

Here's a quick example program demonstrating the combination of https://twistedmatrix.com/documents/17.9.0/api/twisted.logger.LogLevelFilterPredicate.html <https://twistedmatrix.com/documents/17.9.0/api/twisted.logger.LogLevelFilterPredicate.html> and https://twistedmatrix.com/documents/17.9.0/api/twisted.logger.FilteringLogObserver.html <https://twistedmatrix.com/documents/17.9.0/api/twisted.logger.FilteringLogObserver.html>.

import sys
from twisted.logger import (Logger, FilteringLogObserver,
                            LogLevelFilterPredicate, globalLogBeginner,
                            LogLevel, textFileLogObserver)

alog = Logger(namespace="a")
blog = Logger(namespace="b")

predicate = LogLevelFilterPredicate()
predicate.setLogLevelForNamespace("a", LogLevel.debug)
predicate.setLogLevelForNamespace("b", LogLevel.warn)

globalLogBeginner.beginLoggingTo(
    [FilteringLogObserver(textFileLogObserver(sys.stdout), [predicate])]
)

alog.info("one")
blog.info("two")
alog.critical("three")
blog.critical("four")


> If any of these changes sound like they would be useful apart from the
> specific requirements of that application, I could help implement
> something based on what I wrote [0].

We'd certainly be interested in contributions to the logging subsystem!  But I'd probably want to hear a little more about how and why you thought these things were useful first.

-g

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20180105/edaab9dc/attachment-0002.html>