<div class="gmail_quote">On Sat, Jan 22, 2011 at 10:02 PM, Glyph Lefkowitz <span dir="ltr"><<a href="mailto:glyph@twistedmatrix.com">glyph@twistedmatrix.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div style="word-wrap:break-word"><br><div><div class="im"><div>On Jan 22, 2011, at 3:18 PM, Tom Davis wrote:</div><br></div><blockquote type="cite"><div class="gmail_quote"><div class="im">On Fri, Jan 21, 2011 at 10:00 PM, Glyph Lefkowitz <span dir="ltr"><<a href="mailto:glyph@twistedmatrix.com" target="_blank">glyph@twistedmatrix.com</a>></span> wrote:<br>
</div><blockquote class="gmail_quote" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color:rgb(204, 204, 204);border-left-style:solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><div class="im"><div><div>On Jan 20, 2011, at 11:20 PM, Tom Davis wrote:</div><div><font color="#006312"><font color="#144FAE"><br></font></font></div></div></div><div class="im"><div>
If branches that are out there <i>don't</i> meet these standards, commenting on their tickets and getting them deleted or closed as invalid (as appropriate) would be a big help too. Lots of languishing tickets that nobody knows what to do with is not a good thing, and there's plenty of opportunities for interested parties to reopen tickets, attach new patches, and object in various ways, so you shouldn't be too concerned about stepping on toes. Focus is a valuable commodity.</div>
</div></div></div></blockquote><div class="im"><div><br></div><div>I was wondering to what extent it would be helpful to actually reply to all the tickets, or just the ones that seem to have actionable next steps. I will try to find something to ask or opine on in each of the documentation tickets so we can get them moving along or removed.</div>
</div></div></blockquote><div><br></div><div>Getting rid of dead tickets is almost as important as actually getting valid tickets moved along. I think this effort will be hugely valuable.</div></div></div></blockquote><div>
<br></div><div>Will continue commenting on tickets and asserting non-existent power to get them resolved! ;)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div style="word-wrap:break-word"><div><div class="im"><br><blockquote type="cite"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color:rgb(204, 204, 204);border-left-style:solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><div></div><div>Part of my comment about low-hanging fruit was to help you get familiar with and integrated into the development process. Going through the process of getting patches reviewed and accepted will be _much_ easier if you go through the motions of doing a few trivial things first. In fact you may want to just pick up a couple of trivial non-docs patches as well, which might help you on documenting the development process :). <<a href="http://bit.ly/easy-twisted-tickets" target="_blank">bit.ly/easy-twisted-tickets</a>> might help you there.</div>
</div></div></blockquote><div><br></div><div>I will find an easy ticket or few to bang out. I just replied here (<a href="http://twistedmatrix.com/trac/ticket/2491" target="_blank">http://twistedmatrix.com/trac/ticket/2491</a>) in an attempt to get started on one that doesn't already have a long history.</div>
<div><br></div><div>After spending about an hour going over the "easy" tickets, it seems many of them are in odd states. Either they're done and waiting on something undefined or there is an incomplete debate in the comments or the owner disappeared or... well, there are lots of examples. Maybe this is just me being dense or whatever, but I think (at least as a newcomer) I could mass-update all these tickets with "Guidance!" and it would more often than not be a relevant comment given the state of the ticket.</div>
</div></blockquote><div><br></div></div><div>No, this is not you being dense. At least among the core developers in Boston, this is a widely-recognized and frequently-complained-about problem, and it's something I'd like everyone doing ticket reviews to please think about.</div>
<div><br></div><div>Reviewers: if you make a comment on a ticket, but you don't say what you want to happen next, then you have effectively killed progress on that ticket until some other reviewer comes along and contradicts you to get things moving again. This is especially true if you make one trivial comment on the ticket and remove the review keyword, but don't say "please address these issues and then merge" or "please address these issues and then resubmit for review". If you've done a partial review, and made a comment like "I don't like this aspect of the design" or "please update copyright dates" or "your docstring formatting is wrong", please note in your comment that this is not a complete review, and <i>don't</i> remove the review keyword. Removing the keyword will introduce additional latency for the contributor, when other reviewers might still come along and attach more comprehensive feedback. There are few things more discouraging than having one free weekend every six months to work on a ticket, and to come back every time to "oops, you forgot to update the copyright date and insert a blank line in one file of your 300-line patch".</div>
<div><br></div><div>So, Tom: mass updating those tickets wouldn't be helpful, but an update every couple of days with a specific question on one of these I-don't-know-what-to-do tickets would be great. Your question on <a href="http://tm.tl/2491" target="_blank">tm.tl/2491</a> was a definite step forward.</div>
</div></div></blockquote><div><br></div><div>Awesome! I'm sure a lot of this has to do with the fact that once you maintain and work on something like this for a decade you take a lot for granted, especially with regard to your own policies and procedures. It becomes tedious to provide (or just difficult to enumerate) all the details necessary to new people. Hopefully as I become more familiar with them I can ensure some of the tedious/forgotten corners become documented in a way that makes it easier. And just point out when the current status of something is way too ambiguous to be actionable for more than a couple folks who are too busy to deal with it.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div style="word-wrap:break-word"><div><div class="im"><br><blockquote type="cite"><div class="gmail_quote">
<div>Here's a great example of what I'm talking about (and I apologize for the mid-message digression, but I think it's relevant...): <a href="http://twistedmatrix.com/trac/ticket/4636" target="_blank">http://twistedmatrix.com/trac/ticket/4636</a>. This seems totally trivial, but five months later __all__ was never changed in t.i.main and JP's buildbot link is broken. Whether the offending class should be removed from __all__ or imported instead was never even mentioned. Anyway, I took a stab at it and attached a trivial patch so this isn't just another complaint (and it looks like JP closed it while I was drafting this email, so that's great!). But I do agree that working tickets would really help document the process!</div>
</div></blockquote><div><br></div></div><div>As you have discovered here, drawing attention to a ticket in this "stuck" state will often cause it to get un-stuck. So please keep doing that.</div><div class="im">
<br><blockquote type="cite"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color:rgb(204, 204, 204);border-left-style:solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div><div></div><div>Mostly, I really don't want you to write a gigantic pile of new documentation and then find, when you're "done", that you missed some nuance of the coding standard, or the patch is too big to be reasonably reviewed, and that now you have three months of additional work to do before it's all <i>really</i> done. Experience with the process will mitigate that problem significantly. (And in fact I hope that you don't actually have a gigantic pile of stuff to commit all at once at any point, and can continue this work incrementally as a series of small tickets, but I realize that later on some of the index reorganization stuff may need to be big. This is mostly just restating what Kevin already said in his message, but it bears repeating.)</div>
</div></div></blockquote><div><br></div><div>The more I think about this, the more I agree with you. My initial inclination was just to start from scratch and move over existing docs that I found I could use. This doesn't conflict with the reuse I've been supportive of (and for good reason: there's a lot of decent documentation already there) but it does conflict with the Twisted development policies.</div>
<div><br></div><div>At the end of the day, I can't really submit patches to existing documentation until said documentation is in Sphinx form.</div></div></blockquote><div><br></div></div><div>Aaaaaaaauuuuugh.</div><div>
<br></div><div>When you say this, the first thing I think is "okay then, the sphinx migration is now introducing roadblocks into the documentation process and preventing people from contributing documentation, so let's just cancel it". So please don't say things like this :). The whole reason that we insisted on the current process for the sphinx migration was to avoid this kind of blockage.</div>
</div></div></blockquote><div><br></div><div>Sorry! I didn't mean to suggest the migration is a roadblock. It's *inconvenient* that it isn't already done, but it's certainly not stopping anything to the point of cancelation (and doing so would probably do more harm than good since every new document we *can* convert is one that doesn't have to be converted manually)</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div style="word-wrap:break-word"><div><div><br></div><div>You can submit documentation patches right now, in Lore format - which, I would like to remind you, is 99% plain-vanilla HTML and very easy to edit - and get them accepted. You can see on the sphinx buildbot (once a few minor issues are fixed, as discussed earlier in this thread) what those changes will look like once converted to ReST->Sphinx. Or you can simply run the conversion yourself locally - hopefully Kevin will chime in on how to set that up so I don't need to look it up :). As long as you aren't trying to do anything fancy with diagrams or tables (and <i>most</i> of the documentation really should not need elaborate diagrams or tables), you shouldn't run into any issues.</div>
</div></div></blockquote><div><br></div><div>Yes, aside from things like toctrees and glossary references, it should be largely similar (although I tend to get pretty verbose with Sphinx/reST declarations because they're so handy). However, there will likely be times where it is either wildly inefficient or flat out impossible to port something to/from Sphinx.</div>
<div><br></div><div>Specifically, any new/modified documentation that involves messing with code listings is going to be problematic. The current system calls for multiple code listing modules with copious code duplication and a general lack of testability; the new system calls for tested, combined modules who's display can be managed by Sphinx (see: <a href="http://sphinx.pocoo.org/markup/code.html#includes">http://sphinx.pocoo.org/markup/code.html#includes</a>). I'm sort of at a loss for what to do in these situations and no simple solution jumps out at me. Having to make the cognitive switch between "I will write a dozen incremental, untestable listings" and "I will write compact, TDD listings" seems like it could cause a lot of latency (and extra work) regardless of which side of the fence you start on.</div>
<div><br></div><div>Maybe we'll just have to take these sorts of conflicts one at a time and try to make decisions as quickly as possible as to which require backwards compatibility and which don't? *shrug*</div>
<div>
<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div style="word-wrap:break-word"><div><div><br></div><div>Heck, if you want to write your documentation patches as ReST snippets and attach them to tickets, I'm sure you will find <i>many</i> willing contributors (myself included) who will jump in and do the format-munging manually to get them integrated into exiting lore documents so that they can make it to trunk immediately.</div>
</div></div></blockquote><div><br></div><div>That may make sense for some issues. However, I am definitely up for modifying Lore docs when the changes wouldn't create a significant discrepancy between formats that would need to be manually resolved later (see: <a href="http://twistedmatrix.com/trac/ticket/2018">http://twistedmatrix.com/trac/ticket/2018</a>).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div style="word-wrap:break-word"><div><div><br></div><div>Or, you can just jump in with ticket reviews, or breaking up chunk tickets, to move along the Sphinx migration quicker. But "I will work over here in a corner until somehow this gigantic pile of work gets finished by somebody else" is not a good strategy. (I'm not saying that that's exactly what you're proposing to do, but the aspects of what your proposing which align with that may not be the best way to proceed.)</div>
</div></div></blockquote><div><br></div><div>Fair enough. I am trying to split my time equally between current, future, and new docs issues; I spent all day managing/working tickets and emailing, for instance. I definitely don't want to create a separatist thing here. :)</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div style="word-wrap:break-word"><div><div class="im"><br><blockquote type="cite"><div class="gmail_quote">
<div>I guess one way this could work to the happiness of all involved is:</div>
<div><ol><li>Get re-org nailed down; basic samples for whatever pages I think should have standardized format / elements.</li><li>Mixed in here, find existing docs and non-docs tickets to work on trunk.</li><li>When the re-org structure is ready, create a new branch from whatever the current Sphinx conversion branch is (hopefully it'll be "done" at this point)</li>
<li>Migrate new structure to branch, moving the existing docs to semi-logical-but-possibly-temporary places in the new structure.</li><li>Submit tickets / patches against existing documentation in said branch to move docs to new(er) locations, submit new TDD-style code samples, general edits, new docs, etc.</li>
<li>When everything is complete, submit a final ticket to merge to trunk</li></ol><div>Does that make sense or am I still cognitively off the mark in terms of existing development practices?</div></div></div></blockquote>
<div><br></div></div><div>With the caveat of what I said above, most of these things sound like generally good things to do. Except, first, I'd still like to see a ticket for what "the re-org" is actually going to be and what the point of it is. If I've only learned one thing in maintaining Twisted for 10 years, it's that a description of <i>what we are trying to do</i> separate from <i>how we are going to do it</i> early on is essential to make sure that others can give you useful feedback - and, more importantly, provide you with resources. In this case, those resources would be pointers to other areas in the documentation that you may not have noticed which already do some of what you are trying to do, but may be poorly integrated.</div>
</div></div></blockquote><div><br></div><div>Okay. I will formalize the reorganization based on what we've already discussed on here and at the meet-up and hopefully that will make things at least as clear as mud! I have an official one for the new howto/Task standardization already (<a href="http://twistedmatrix.com/trac/ticket/4818">http://twistedmatrix.com/trac/ticket/4818</a>), let me know if it's at all useful and/or what you mean here.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div style="word-wrap:break-word"><div><div class="im"><div><br></div><blockquote type="cite"><div class="gmail_quote">
<div>I will be fleshing out those goals more in the coming days, but some of it is implicit within the new documentation structure that's already in my repo.</div></div></blockquote><div><br></div></div><div>$ python -c 'import this' | grep Explicit # :-)</div>
<div class="im"><br><blockquote type="cite"><div class="gmail_quote"><div>As for a statement of goals when replacing/removing specific documentation, sure, I can do that as necessary. It's impossible to completely remove the element of taste but I will certainly avoid the "because I like this better" argument for making changes.</div>
</div></blockquote><div><br></div></div><div>Again, this is more about having a good statement of purpose up front, so that everyone is focused on the same thing, than having an argument that you can trot out later once everyone starts nitpicking from a hundred different perspectives :). Plus, such a statement of purpose can serve as a focus for you as well.</div>
<div class="im"><br><blockquote type="cite"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color:rgb(204, 204, 204);border-left-style:solid;padding-left:1ex">
<div style="word-wrap:break-word"><div>
<div><blockquote type="cite"><div><div><blockquote class="gmail_quote" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color:rgb(204, 204, 204);border-left-style:solid;padding-left:1ex">
<span style="font-family:arial, sans-serif;font-size:13px;border-collapse:collapse">You will probably have to press us core developers on this one, and you may spark some debates. </span><span style="font-family:arial, sans-serif;font-size:13px;border-collapse:collapse">These tend to sputter out with no clear resolution as everyone is frustrated that nobody's solution (not even their own) is ideal, but you would be doing us all a great service if you really forced us to develop a consensus about certain things (like "what's the best way to build a twisted command-line program", for example) and agree to agree on the current documented "best practice" for those things.</span></blockquote>
<div><br></div><div>Debates are great!</div></div></div></blockquote><div><br></div></div><div>Debates <i>that reach some kind of conclusion</i> are great :). Debates that just go in circles until everybody feels crappy about the topic aren't. So I'm really just asking you to help us make these debates into the great kind. (The rest of your reply seems to agree with that, I just wanted to be clear.)</div>
</div></div></blockquote><div><br></div><div>Yeah, as I've already mentioned in this reply, the "limbo syndrome" of many tickets drives me nuts. I'm sure you guys hate it too, but having somebody new around who isn't afraid to ruffle some feathers for the greater good and isn't yet at some "acceptance" level with it should be helpful! That being said, I will do my best not to be a jerk about it; you guys have been doing this for much, much longer and I respect that.</div>
</div></blockquote><div><br></div></div><div>Tickets in limbo need to be eliminated. I think you will find that just about everyone is happy to pick up these discussions with an injection of a little bit of attention and energy from a contributor. We all <i>know</i> that these sap the will of new contributors and thereby decrease the overall pool of available time, so I think you may have to try really, really hard to actually ruffle any feathers :).</div>
</div></div></blockquote><div><br></div><div>Great, I was hoping I wasn't the only person who saw them this way!</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div style="word-wrap:break-word"><div><div class="im"><br><blockquote type="cite"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color:rgb(204, 204, 204);border-left-style:solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><div><blockquote type="cite"><div><div><div>If you like Python and DNS or SMTP or whatever the hell else, what has the potential to be a more awesome implementation than its Twisted one? Let's help people find out for themselves that the answer is <i>nothing</i>. Then they'll want it to be <i>their</i> project.</div>
</div></div></blockquote><div><br></div></div><div>If you do this, you will be my hero forever.</div></div></div></blockquote><div><br></div><div>Twisted is a naturally-superior Python choice for most supported protocols by virtue of the fact that most network "stuff" benefits from event-based solutions. Like, find me a Python DNS server that <i>isn't</i> implemented in Twisted. People have these weird misconceptions about Twisted; it's really hard to grasp, ugly, unmaintained, etc. Ultimately, I want to correct these misconceptions, have <i>twisted.names</i> show up first on Google for "Python DNS Server", and all the other shit that really should be the case but isn't. Resolving these issues should ultimately resolve the problem of maintainers, community involvement, etc.</div>
</div></blockquote><div><br></div></div><div>+10000.</div><div class="im"><br><blockquote type="cite"><div class="gmail_quote"><div>I can't promise that in the course of this crusade I won't cause grief by trying to change things, but I will always strive to have a rational reason for wanting to do so!</div>
</div></blockquote><br></div></div><div>Please keep the grief coming! It'll all be worth it, I'm sure :). Everything that you've said so far that I disagree with has just been an opportunity to communicate some of the Twisted development philosophy which may not yet be written down - and should probably be part of the documentation eventually. So you have yet to do anything which has put me out even in the slightest. We need more new contributors feeling comfortable being bold and trying to Just Do It.</div>
</div></blockquote><div><br></div><div>That's good to hear! This is probably a discussion for another thread, but Twisted has always seemed a bit imposing in terms of ease of contribution. That may be a public relations issue as much as a procedural one, but making contribution a relatively frictionless, step-by-step process will go a looooong way to getting more people to do it.</div>
<div><br></div><div>I also think that reviewer conduct is as important as the tone of the documentation (which should probably start with something like "These are Twisted's contributor rules and processes. We urge new contributors to start by simply writing a patch, though."); JP helped me out earlier today (<a href="http://twistedmatrix.com/trac/ticket/4636">http://twistedmatrix.com/trac/ticket/4636</a>) by accepting my patch despite a couple missing details. That gave me a nice sense of accomplishment without feeling like it was diluted by a few trivial details. I think there is *very* little cost to these sorts of things because either you just got (a) a one-time contribution you wouldn't have with a dozen steps, or (b) a new long-term contributor thanks to quick feedback, appreciation, and flexibility on the part of the team. But I'm drifting now, so shall drift in the direction of bed and pick things up tomorrow...</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div style="word-wrap:break-word"><div><br></div></div><br>_______________________________________________<br>
Twisted-Python mailing list<br>
<a href="mailto:Twisted-Python@twistedmatrix.com">Twisted-Python@twistedmatrix.com</a><br>
<a href="http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python" target="_blank">http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python</a><br>
<br></blockquote></div><br>