[Twisted-web] Thoughts on documentation, wading into Twisted, Nevow, etc.

[Terry Jones]

Here's a summary of thoughts I've had over the last month or so. Please try
to take the following as positive / constructive. I'm happy to be involved
in trying to alleviate what some people may agree are problems.  Or people
may not agree that they're problems, which is fine too - more on this
below.

A few months ago I started looking at various (mainly Python) frameworks
with an eye to using one for a project I'm working on. When I found Twisted
(and friends) I was really happy. I've been writing code for what seems
like a long time (30 years) and I think I'm pretty critical and demanding
when it comes to good and clean design. I certainly had not expected to
find something like Twisted and its various additional components. Without
having written a line of Twisted code, I made the decision to throw away a
full year of my own low-level, efficient, debugged, C code (networking,
custom RPC, argument marshaling, tons of sweat). That was a pretty serious
step, not taken lightly, and, I still think, a good one.

That's why I'm still here, spending hours writing this email, wanting to
press ahead, and willing to put in time and effort to help.

In one corner then, we have a great foundation - elegant, dynamic, well
designed, full of potential, nice programming language, written by talented
and responsive developers, etc. But... in the other corner, documentation
that very often seems to be a complete shambles.

There, I've said it.

I don't claim that this is actually a problem. That depends. It may well be
that I've happened along at a moment which is not optimal for my needs and
abilities; that there is a well-organized design, development, and
documentation trajectory; and that I just happened to arrive at a bad point
on that curve.

I fully sympathize with things being less well documented than they could
otherwise be.  I love writing code. Following the rise of highlighting
editors I've even become quite fond of comments. But I'm not even slightly
fond of writing external docs. I think an important part of getting things
right in design is to implement something that feels right and then rework
it until it really feels right and clean, and is also useful. There's
little point in documenting things before they have settled. Writing code,
debugging, re-designing, building apps on top of libraries - all those
things, to me, should take precedence over documentation. I don't believe
in writing fully fleshed-out specs, docs, and tests ahead of
implementation.  I even have little time or sympathy for people (like
myself in this case) who don't have what it takes to use emerging tools
before they're fully documented for use. I've spent plenty of my own free
time in the last 20 years writing, giving away, and supporting, code I felt
like producing.  So I think I understand the equation pretty well.

But, Twisted and its related components are released. Valuable (to the
project) people are trying to use them. This can be frustrating. I think I
have a reasonably high frustration threshold (but maybe I'm wrong - maybe
I'm just old & cranky).

It's quite general to talk about "Twisted and its related components".
There are many moving parts (some conveniently accompanied by moving
names).  So let me just pick on Nevow - it's perhaps in the middle in terms
of maturity, documentation, frustration level, etc. I actually don't mean
to single out Nevow in particular, but it has qualities that make it a good
example target for discussion. Apologies in advance to anyone this seems to
pick on - that's not my intention, I have no idea who wrote (or didn't
write) some of this stuff. I know Matt is involved as he's been kind enough
to several times send me long explanations and code in email.

Again, I'm not claiming there's a problem here.

I find Nevow to be extremely promising and yet quite cryptic. Sadly, I've
built dozens of web sites, many of them commercial, done under contracts
with real deadlines, budgets, staff, etc. I hate editing ugly templates in
awkward ugly non-code file formats. There's just so much disgusting stuff
out there. So to see Nevow and Stan is a real pleasure.  But to try to
understand it... argh. Hop onto google and try "nevow documentation" and
read through some other people's experiences (e.g.
http://www.third-bit.com/pywebblog/archives/cat_twisted.html)

The recent "nevow documentation" thread (kicked off by Manlio, at
http://twistedmatrix.com/pipermail/twisted-web/2006-May/002652.html) is
mainly illustrative for its lack of conclusion. I don't find the suggested
page at http://divmod.org/trac/wiki/TitleIndex to have much (if anything)
to do with documentation - it seems to be links to code. It's not clear if
the wiki is more a help or a hindrance (this is probably too condemning,
but after spending more than enough time looking at things that one later
discovers are out of date, replaced, deprecated, etc., you start to wonder
about everything).

Nevow comes with many examples, and that's great. But they're still
cryptic. E.g., a grep url\\. */*.py in Nevow's examples directory gets me
over 30 hits, but I don't see that class documented anywhere and when I try
to use it in my code it doesn't work. E.g., formless is used in many
examples, but is apparently a dead end. If you disagree with that, you'll
be reinforcing my sentiment that things are a bit of a mess :-).

I can give more examples. There's quite a bit that seems very cryptic, and
I mean beyond the things described in the already very terse and piecemeal
Nevow documentation.  Of course it takes time to spell things out, but if
people want them I will. I already decided to take the evening off, have a
couple of beers, and write this email.

Before trying to finish on a politically correct, upbeat, & constructive
note, here's a question:

How was it that Twisted (or just Nevow?) managed to lose the attentions of
Kieran Holland? He (she?) was clearly into Nevow, put a lot of effort into
understanding things, and yet headed off to Django, which, reading between
the lines, he/she didn't really think was even nearly up to the
Twisted/Nevow standard.  That's a bad loss. People who can read and write
code and who are willing to spend serious amounts of time helping ease
others into projects are so valuable. The Addendum of Kieran's Stan page
(http://www.kieranholland.com/code/documentation/nevow-stan/) seems to
spell things out pretty clearly. It feels like the Nevow (and broader?)
community has done itself a serious disservice in losing Kieran's attention
(but, see above comments on whether this is perceived as a problem,
trajectories, etc).

To me, totally on the outside, I do not get the impression that there is a
clear design, code, and release trajectory in place. It feels like there is
a bunch of very talented designers and coders who are banging out an
increasing number of cool projects at high speed, leaving a frustratingly
incomplete, misleading, and atrophying breadcrumb trail of documentation
behind them.


On to another question: do people think there a problem here, or not?  The
answer may be no.  I think that's fine, I'm happy to be told to read the
code and to shut up. At least that would be definitive, and not at all
unwelcome.

But, if you think there is a problem, what can we do? Here are some
possible concrete steps that wouldn't take huge amounts of time.

 1) Go through all Twisted & friends web sites and wiki and _ruthlessly
    delete everything_ that mentions obsolete, deprecated, or renamed
    projects / modules, broken links, example code using such, etc. There
    are many of these things, and they are highly misleading and
    frustrating and give a bad overall impression.

 2) State, where appropriate, that the source code is currently the best
    documentation. There's no shame in this, and it's much better than
    having people run across and waste time on obsolete docs, or spend time
    looking for things that do not exist. E.g., "At this point the
    nevow.url class is totally undocumented. See examples/{x,y,z} for
    example usage and the code and comments in nevow/url.py for the best
    current documentation."

 3) Have someone, preferably an outsider (could be me), write up and
    maintain a no-punches-pulled overview document about using the various
    Twisted components. Stick a link to this in a prominent place.
    http://twistedmatrix.com/trac/wiki/WebDevelopmentWithTwisted is a good
    start.

 4) Have someone go through example code to stick in more comments, get rid
    of old stuff (no need to rewrite if that takes too long, just throw it
    away). E.g., the widespread use of formless in Nevow examples. Is it
    in, is it out?  If the latter, get rid of it before more people die.

 5) Have someone do some web searches for stuff known to be fixed or out of
    date, and send mail to the authors asking if they'd please consider
    updating or otherwise clarifying what they've written, could they add a
    link or a postscript, to help out future people looking to use Twisted
    & friends? No need to try to hide the fact that things were a bit of a
    mess - just aim not to have people get an out-of-date impression or
    follow links to incorrect / obsolete pages (e.g., links to nevow.com).

I could probably come up with more suggestions. Given the inclinations of
(I assume) many people still reading this message, no-one is too much
inclined towards producing actual documentation (and it may not be the
right moment, as above). So it makes sense to focus on easy hits: purging
old crud, minimizing wasting the time of people looking to get involved,
making it clear what the barriers are (e.g., point people at the source,
tell them that at this point the only way to understand is to ask questions
(but time is precious) and to read the code).

OK, enough words for now. Thanks again for all the help, all the code, etc.

Terry