[Twisted-Python] [ANN] structlog – bring context & structure to your logs without boilerplate
Hynek Schlawack
hs at ox.cx
Tue Sep 17 10:12:01 MDT 2013
Hi folks,
today, I’ve release the 0.2.0 of my structlog project whose ambition is no less than changing the way we log in Python. :)
And since it sports dedicated support for Twisted <http://www.structlog.org/en/latest/twisted.html> (but is not limited to it, it wraps *any* logger), I’d like to introduce you to it.
Basically, its premise is that events happen in a context and you want to log out both effortlessly. Nowadays logs are usually consumed by parsers anyway, so regular, easily parsable logs are a good thing™.
This is a nice example of a Twisted application showing off a bit what structlog is capable of:
import sys
import uuid
import structlog
import twisted
from twisted.internet import protocol, reactor
logger = structlog.getLogger()
class Counter(object):
i = 0
def inc(self):
self.i += 1
def __repr__(self):
return str(self.i)
class Echo(protocol.Protocol):
def connectionMade(self):
self._counter = Counter()
self._log = logger.new(
connection_id=str(uuid.uuid4()),
peer=self.transport.getPeer().host,
count=self._counter,
)
def dataReceived(self, data):
self._counter.inc()
log = self._log.bind(data=data)
self.transport.write(data)
log.msg('echoed data!')
if __name__ == "__main__":
structlog.configure(
processors=[structlog.twisted.EventAdapter()],
logger_factory=structlog.twisted.LoggerFactory(),
)
twisted.python.log.startLogging(sys.stderr)
reactor.listenTCP(1234, protocol.Factory.forProtocol(Echo))
reactor.run()
It will give you an output like:
2013-09-17 17:40:23+0200 [-] Log opened.
2013-09-17 17:40:23+0200 [-] Factory starting on 1234
2013-09-17 17:40:23+0200 [-] Starting factory <twisted.internet.protocol.Factory instance at 0x108301488>
2013-09-17 17:40:28+0200 [Echo,0,127.0.0.1] peer='127.0.0.1' count=1 connection_id='4e2ee31c-b3ff-478e-ae06-7b1a492fce45' data='foo\n' event='echoed data!'
2013-09-17 17:40:33+0200 [Echo,0,127.0.0.1] peer='127.0.0.1' count=2 connection_id='4e2ee31c-b3ff-478e-ae06-7b1a492fce45' data='bar\n' event='echoed data!'
2013-09-17 17:40:44+0200 [Echo,1,127.0.0.1] peer='127.0.0.1' count=1 connection_id='3118f673-482b-471f-a206-e8f97f9a9c2c' data='qux\n' event='echoed data!'
2013-09-17 17:40:53+0200 [Echo,0,127.0.0.1] peer='127.0.0.1' count=3 connection_id='4e2ee31c-b3ff-478e-ae06-7b1a492fce45' data='twisted prevails\n' event='echoed data!'
(you can find this and more examples at <http://www.structlog.org/en/latest/examples.html>)
In short:
- It allows you to build context by binding values to loggers. This context is just a dictionary. Once you log an event out (again, arbitrary number of key/value pairs), it gets merged with the context, processed by configurable processor chain and passed to your original logger.
- Your loggers are immutable (by default infidels can use thread local storage) and you’ll get a new one on each binding. Immutable local data is awesome.
- You can define processors that can mangle, filter, and format your log entries.
- Configure once, then just call getLogger (which is a Twisted-friendly alias for get_logger) in regular code.
- Of course there is a JSON renderer built right in – just tell log stash about it and be merry.
*Please* have a look at <http://www.structlog.org/en/0.2.0-0/getting-started.html>, I don’t want to reproduce it here. :)
Credit where credit is due: I picked up the idea of bound loggers from David Reid’s work on otter <https://github.com/rackerlabs/otter/tree/master/otter/log> which in turn was based on ideas by JP (I’ve been told).
You can find everything you need to know at http://www.structlog.org/ – I’d be happy about any constructive feedback and even more so about contributions!
Cheers,
Hynek
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130917/6641f382/attachment.pgp>
More information about the Twisted-Python
mailing list