[Twisted-Python] Lore to Sphinx Conversion Progress Report

Glyph Lefkowitz glyph at twistedmatrix.com
Thu Dec 3 04:00:48 EST 2009

On Dec 3, 2009, at 1:29 AM, Kevin Horn wrote:

> On Wed, Dec 2, 2009 at 8:38 PM, <exarkun at twistedmatrix.com> wrote:
> As far as the (I'm going to call it) roadmap goes, the thought that's
> pushed its way to the fore for me is that I'd like to try this with
> something smaller and simpler than Twisted first.  It would be nice if
> the Divmod projects would qualify here, but they may not be active
> enough for any real experience to accumulate.

> I haven't run my lore2sphinx script against the Divmod stuff yet, but I could certainly try it.  The intention is to convert the Divmod stuff as well as the Twisted stuff eventually though.
> I've been concentrating on the Twisted docs, since they seemed the highest priority, but I could detour a bit if you like.  Obviously that would slow progress on the Twisted stuff a little.

While converting pyOpenSSL's documentation would be a worthwhile goal in itself, I don't actually like this idea.

Mainly I disagree with the premise that accumulation of experience is necessary for the conversion.  But this is not a strong disagreement, as I don't understand the motivation for saying so in the first place :).

The beauty of this plan, as far as I see it, is that the only person who has to get any significant experience with Sphinx in order for the conversion to happen is Kevin.   Mostly what the rest of us will do is read the documentation and make sure it looks OK.  Obviously we all need to learn ReST *after* that conversion to write documentation, but there is so much documentation of ReST and sphinx available that I'm not really worried about that part.

More importantly, as JP already noted, the pyOpenSSL documentation is in a different format and the conversion would use a different toolchain, so even if we do have to get some practice, it's not a particularly helpful place to start.  If we *do* need practice for some reason, I think Nevow would be the best place to start, but then, I don't see why the activity level of the project makes any difference.

I have a few issues with the roadmap too, though:

"blah, blah": I'm not sure what that's supposed to mean.

"branches containing lore docs changes should be separated into two groups..." - no criteria are specified for deciding which go into which group.  Ideally we could just get all of those changes merged; if you are making progress on the conversion I'm sure we could organize a sprint to evaluate those changes and either abandon them or get them into review.

Identifying tickets which propose lore functionality is pretty easy; just look at <http://twistedmatrix.com/trac/query?status=new&status=assigned&status=reopened&component=lore&order=priority>.  Any tickets which have not been classified properly and therefore don't show up in that list can be closed later, as we discover them.  I strongly feel that we do *not* need to conduct an exhaustive review of the entire ticket tracker and get everything perfectly in order in order to do this conversion, as long as it's clear to everyone what is supposed to happen to lore-related tickets in the future.

I see the biggest risk at the "docs freeze" step, that the doc conversion guy (or team, as the case may be by that point) will start work, then get distracted and walk away for 6 months, leaving a long period of time where nobody is supposed to write or edit documentation.

The whole point of our branch-based review process is to avoid this sort of situation.  We can't always avoid it (for example, the immense outstanding Conch branches that made everyone afraid to edit those warning-filled tests for years) but I think it's best to follow the same plan as for any branch, and have no formal "freeze" duration, just a point where the conversion branch gets merged to trunk.  It's OK if a few stale doc branches get left out in the cold during the conversion; if they're still stale once the conversion is ready, they must not be terribly actively worked on anyway.

Even in the worst case, where a branch is left stale well after the docs have been deleted from trunk, presumably the author of the change could run lore2sphinx against the conversion, copy the sphinxified doc to their trunk working copy, and make a new branch.  Assuming that the output of the conversion tool is deterministic, the diff should be small and readable.

I'm not saying that we shouldn't identify those changes... It would be good to identify the branches with outstanding doc changes so that we would at least *know* how many changes will be broken, and perhaps motivate their authors to fix them ahead of time.

I'm mainly interested in the "phase 0" outlined in the roadmap; I think that the stuff for phase 1 and 2 sounds good, but I don't think we need any special planning for it, since it fits into our normal development workflow pretty neatly.  People will be filing bugs for documentation typos pretty much forever ;-).  I'd like it if you could break down the "phase 0" a bit more clearly with regard to what happens when, since the review *before* the Big Switch gets thrown to put these changes in trunk is the most important part to get timely feedback from the community.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20091203/1e0b1503/attachment.htm 

More information about the Twisted-Python mailing list