<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Mar 31, 2012, at 6:10 PM, Brian Warner wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>On 3/30/12 5:49 PM, Glyph wrote:<br><br><blockquote type="cite"><blockquote type="cite"><a href="http://foolscap.lothar.com/docs/logging.html">http://foolscap.lothar.com/docs/logging.html</a> has details.<br></blockquote></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Indeed, Foolscap has informed many of my future wishes for Twisted<br></blockquote><blockquote type="cite">logging, especially its incident reporting and logport features. (I<br></blockquote><blockquote type="cite">wasn't aware of UMIDs but I have independently invented the same<br></blockquote><blockquote type="cite">thing.)<br></blockquote><br>(actually, the "UMID" support consists of ignoring extra kwargs, and an<br>emacs macro which inserts umid="RANDOM" at point. It works remarkably<br>well, and is a lot cheaper than having log.msg() capture the call stack<br>so you can figure out which copy of the otherwise-anonymous<br>log.msg("done") calls was invoked this time)<br></div></blockquote><div><br></div><div>I was thinking more along the lines of something that would insert the output of "python -c 'import uuid, sys; sys.stdout.write(str(uuid.uuid4()))'", but yeah, same general idea. (Why do you have to ignore extra kwargs?)</div><br><blockquote type="cite"><div><blockquote type="cite">Could we perhaps get some of foolscap logging's core features into<br></blockquote><blockquote type="cite">Twisted? We could have an AMP interface for subscribing to log streams<br></blockquote><blockquote type="cite">that would be slightly more low-fidelity (but, I would guess,<br></blockquote><blockquote type="cite">higher-performance) than the Foolscap one, but work based on the same<br></blockquote><blockquote type="cite">core mechanism which foolscap could then subscribe to.<br></blockquote><br>Yeah, that'd be great. An AMP interface sounds lovely. I'd have to think<br>about potential gotchas, but I'm sure we can work them out. Foolscap's<br>remote protocol relies upon all the log.msg() arguments being<br>foolscap-serializable, which occasionally reveals places where we're<br>accidentally passing object instances into the log. (it uses<br>CopiedFailure a lot, so we'd need to come up with an AMP equivalent).<br>The stored-on-disk format just uses pickle, which I kinda regret, but it<br>should be pretty easy to replace that with something safer.<br></div></blockquote><div><br></div><div>I was thinking that AMP would offer an AnyOf() argument type, which would be dynamic within a small set, with a fallback. Within the context of logging, we'd support unicode/bytes/float/int (and maybe some more esoteric numeric tower stuff too, like decimals and fractions), and possibly lists and dicts of those same types (but not structures of structures, so we don't have to worry about backreferences). Maybe tuples supported the same as lists.</div><div><br></div><div>The fallback would simply be "repr() this thing", so if you log an object you still get something somewhat useful without potentially arbitrary serialization.</div><div><br></div><div>Foolscap could take the same data and just make a </div><div><br></div><blockquote type="cite"><div>There's also the notion of a "log port" (a foolscap object with a<br>remote_subscribe() method). Does AMP have any notion of object<br>references?</div></blockquote><div><br></div><div>No, but I don't think we need such a notion. If we did need it for some reason, AMP does have a notion of establishing new message streams over the same connection. This message finally prompted me to file the ticket that I always think was already filed: <<a href="http://twistedmatrix.com/trac/ticket/5587">http://twistedmatrix.com/trac/ticket/5587</a>>.</div><br><blockquote type="cite"><div>If not, I can see how you could call in and ask<br>(synchronously) for the current event log, but not how you'd provide a<br>"callback object" to which future events should be streamed. How does<br>pubsub work in AMP?<br></div></blockquote><div><br></div><div>You just know that your peer knows how to respond to certain commands, and you send those commands. For example,</div><div><br></div></div><blockquote class="webkit-indent-blockquote" style="margin: 0 0 0 40px; border: none; padding: 0px;"><div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "><span style="color: #0c8e12"><b>from</b></span> <span style="color: #033efc"><b>twisted.protocols.amp</b></span> <span style="color: #0c8e12"><b>import</b></span> Command, String, AMP</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "><span style="color: #0c8e12"><b>class</b></span> <span style="color: #033efc"><b>Pub</b></span>(Command):</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "> arguments <span style="color: #797979">=</span> [(<span style="color: #c53530">"channel"</span>, String()), (<span style="color: #c53530">"event"</span>, String())]</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "><span style="color: #0c8e12"><b>class</b></span> <span style="color: #033efc"><b>Sub</b></span>(Command):</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "> arguments <span style="color: #797979">=</span> [(<span style="color: #c53530">"channel"</span>, String())]</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "><span style="color: #0c8e12"><b>class</b></span> <span style="color: #033efc"><b>Hub</b></span>(AMP):</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; color: rgb(184, 81, 252); "><span style="color: #000000"> </span>@Sub.responder</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "> <span style="color: #0c8e12"><b>def</b></span> <span style="color: #033efc">sub</span>(<span style="color: #0c8e12">self</span>, channel):</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "> <span style="color: #0c8e12">self</span><span style="color: #797979">.</span>factory<span style="color: #797979">.</span>observers[channel]<span style="color: #797979">.</span>append(<span style="color: #0c8e12">self</span>)</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; color: rgb(184, 81, 252); "><span style="color: #000000"> </span>@Pub.responder</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "> <span style="color: #0c8e12"><b>def</b></span> <span style="color: #033efc">pub</span>(<span style="color: #0c8e12">self</span>, channel, event):</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "> <span style="color: #0c8e12"><b>for</b></span> observer <span style="color: #b851fc"><b>in</b></span> <span style="color: #0c8e12">self</span><span style="color: #797979">.</span>factory<span style="color: #797979">.</span>observers[channel]:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; font: normal normal normal 12px/normal Courier; "> observer<span style="color: #797979">.</span>callRemote(Pub, channel<span style="color: #797979">=</span>channel, event<span style="color: #797979">=</span>event)</div></div></div></blockquote><div><br></div><div>Then your peer just has the same definition of Pub and has a @Pub.responder that does whatever they like. (Eric Mangold was working on formalizing this process somewhat at the Twisted sprint.)</div><div><br><blockquote type="cite"><div><blockquote type="cite">I suspect that this would be useful to foolscap as well, because then<br></blockquote><blockquote type="cite">the code that foolscap calls into (for example, the web stuff for its<br></blockquote><blockquote type="cite">UIs) would be using the same logging convention.<br></blockquote><br>Yeah, I'd love to get that factored out and let other people hack on it.<br>My web UI skills are laughable.<br></div></blockquote><br></div><div>Actually I was saying that Twisted could provide the core functionality and Foolscap could still provide the web UI - but if you wanted to break things out even further, but I suppose this would also present an opportunity for some third party to plug in some web UI on the side of Twisted that was not necessarily related to Foolscap, which would be cool too.</div><div><br></div><div>-glyph</div></body></html>