[Twisted-Python] the good, the log, and the ugly

Brian Warner warner at lothar.com
Fri Mar 30 15:23:28 EDT 2012

On 3/25/12 7:11 PM, Glyph wrote:

> I'm going to try to get started on record my desired behavior for
> logging as tickets soon though, so Twisted can at least have some really
> good logging tools /eventually/.  

While you're thinking about this space, I'll plug the logging tools I
built into Foolscap (and applications that use it, like Tahoe). I'm sure
many others have built things like it, but some of the features that
I've found particularly useful in Tahoe development are:

 * structured log messages, stored as dicts with non-serialized
 * "umids": unique message identifiers, to find the call site
 * size-limited per-(priority*section) circular event buffers
 * parent/child event relationships, for event trees
 * "Incidents": when an UNUSUAL event is logged, a pickle of the event
   buffers is saved to disk, for later inspection, including events from
   post-trigger error-recovery code
 * "logport": Foolscap interface for subscribing to event streams
 * "flogtool tail" to connect to remote app and watch the event stream
 * web- or text- based event renderers, with expand/hide-subtree buttons
 * "incident gatherer": subscribes to Incident reports from multiple
   remote apps, classifies reports them according to cause

It's inspired by twisted.python.log, of course, and the log.msg()
interface should look pretty familiar. The Foolscap buy-in may be too
much for most apps, but it'd be interesting to find a subset that
doesn't require the networking parts. I tried to figure out how to make
it fit with Python's stdlib logging module, but eventually gave up (I
think the biggest issue was needing to return an event number from the
log.msg() call, for the parent/child stuff).

http://foolscap.lothar.com/docs/logging.html has details.


More information about the Twisted-Python mailing list