[Twisted-Python] the good, the log, and the ugly

Brian Warner warner at lothar.com
Sat Mar 31 18:10:19 EDT 2012

On 3/30/12 5:49 PM, Glyph wrote:

>> http://foolscap.lothar.com/docs/logging.html has details.
> Indeed, Foolscap has informed many of my future wishes for Twisted
> logging, especially its incident reporting and logport features. (I
> wasn't aware of UMIDs but I have independently invented the same
> thing.)

(actually, the "UMID" support consists of ignoring extra kwargs, and an
emacs macro which inserts umid="RANDOM" at point. It works remarkably
well, and is a lot cheaper than having log.msg() capture the call stack
so you can figure out which copy of the otherwise-anonymous
log.msg("done") calls was invoked this time)

> Could we perhaps get some of foolscap logging's core features into
> Twisted? We could have an AMP interface for subscribing to log streams
> that would be slightly more low-fidelity (but, I would guess,
> higher-performance) than the Foolscap one, but work based on the same
> core mechanism which foolscap could then subscribe to.

Yeah, that'd be great. An AMP interface sounds lovely. I'd have to think
about potential gotchas, but I'm sure we can work them out. Foolscap's
remote protocol relies upon all the log.msg() arguments being
foolscap-serializable, which occasionally reveals places where we're
accidentally passing object instances into the log. (it uses
CopiedFailure a lot, so we'd need to come up with an AMP equivalent).
The stored-on-disk format just uses pickle, which I kinda regret, but it
should be pretty easy to replace that with something safer.

There's also the notion of a "log port" (a foolscap object with a
remote_subscribe() method). Does AMP have any notion of object
references? If not, I can see how you could call in and ask
(synchronously) for the current event log, but not how you'd provide a
"callback object" to which future events should be streamed. How does
pubsub work in AMP?

> I suspect that this would be useful to foolscap as well, because then
> the code that foolscap calls into (for example, the web stuff for its
> UIs) would be using the same logging convention.

Yeah, I'd love to get that factored out and let other people hack on it.
My web UI skills are laughable.


More information about the Twisted-Python mailing list