[Twisted-Python] Refactoring Documentation

Glyph Lefkowitz glyph at twistedmatrix.com
Fri Jan 21 21:42:40 EST 2011


I have a lot of input here, but as the guys actually doing the work, the final say is mostly with you and Tom.  So, please feel free to take everything here with a grain of salt.

On Jan 20, 2011, at 2:58 PM, Kevin Horn wrote:

> Here's some things I had "planned" (yes, I'm using the term loosely) to try and add/improve/fix in the Twisted docs after the Sphinx conversion was finished:
> (urg, I know I've been keeping a list of these someplace, and some are in trac tickets, etc. but I can't find it right now, so I may ramble a bit)

Thanks for writing up this list.  I'm re-ordering just a little bit, to start with what I think are the good bits...

> - Task-based docs - how to serve a web page, how to send a mail, how to write an IRC client (just to cut down on the questions!) etc. The basic stuff at first (what newbie's will be looking for), maybe eventually turning into a "cookbook" of sorts.

Yes, yes, yes.  I think everything should be driven from this, as I will repeat many times through this email :).

> - "How to test your Twisted apps" (e.g., the idea of faking up a transport never occurred to me until I read a test that did it, and it's been one of the most useful techniques I've found)

And this is clearly my second priority.

Personally, I really wish there were a tutorial to Twisted itself which were test-driven, rather than separating out "here's how you get started" and "here's how you write your tests".  It may be too much material to present at once, but on the other hand, the trial command-line is very friendly, so that 

> -  Add a basic intro to Twisted.  This would introduce some basic ideas similar to the krondoblog tutorial, but in less detail.

This is where everybody wants to start, but I'd like to play devil's advocate here.  Before anybody tries another grand restart on tutorial / introductory docs (the last one was the now frequently vilified Finger Tutorial), we should ask these questions:

Who wants to read this introduction?
What do they want to learn?
What problem are they actually trying to solve, and how is that going to be furthered by their reading this document?

I think the place to start is really to focus on particular tasks that people want to accomplish, with a pointer to more abstract documentation once they want to learn more about how it's all put together.

Anyway, I'm not going to _completely_ repeat everything I said to Tom at the meetup, but we should take a look at the individual project pages and make them things people want to read, and have tutorials that are focused on each project, since almost everyone who is approaching Twisted as a newbie really wants to get some specific task done, not learn generally about event-driven networking.

> - "What is Twisted good for"

Again: it depends what you want to use it for.  One sprawling page that explains every benefit that Twisted has is really hard to articulate.

> - Explain the most important parts of Twisted for people to get started with.  IMO, this is a) the reactor, b) deferreds, c) some of the basic interfaces/classes, esp. Factories/Protocols
>   (I'd like to see some docs on Transports as well)

... okay I'm totally harping on this now, but:

I think that focusing on the higher-level stuff and getting people running with actual applications should be more of a focus.  There's already a lot of conceptual / introductory material, but it lacks a coherent narrative or clearly-defined audience, and I think that's why it's bad.

In some places this may require some actual code changes.  For example, twisted.names doesn't really have an API to speak of, and while writing a tutorial on how to use it, you're going to bump into that.  That might motivate somebody to go and write the 20 lines of glue which would actually be required to write a dynamic DNS server without subclassing some internal stuff and twiddling undocumented attributes, and that would be great.  (But, lest I get ahead of myself: let's get started on the documentation first; starting off with a bunch of features because "maybe they'll be useful for docs" is an even worse trap to fall into :))

But, I don't think a lot of people are actually developing custom protocols from scratch; and those who are, should probably be using a construction kit like AMP, or PB, or Foolscap, so that they can skip over the tedious and easy-to-get-wrong bits about framing and figuring out a language of how to document messages.

> - more/better UDP stuff

Let's please make this the lowest priority and the last thing that gets done.  I don't think I've ever heard anyone who _actually_ needed to use UDP asking questions about it on IRC or on the mailing list.  If you're using Twisted for DNS (arguably the most common usage of UDP), it's already done for you.  Almost universally, questions about UDP come from people who don't really understand that they should be using TCP because they read on some MMORPG-design forum somewhere that UDP is "faster".

People who already understand the nuances of UDP and know why it's reasonable for movement packets but not for in-game micropayments, for example, might hit a speedbump or two, but they'll get through it pretty easily.  Let's address the audience that really needs the documentation before we start adding it for people whose lives will only get very slightly easier.


> - A revamped section on "How to write twisted documentation", since that would likely be very different after the Sphinx conversion ( What Sphinx markup to use, and I also have some custom extensions, etc. which need to be documented).
> - "How to _build_ the documentation"

Writing this alongside designing the new documentation build process would be a big help :).

(Although really, it should be at most three steps: "install sphinx", "install epydoc", and then "run ./admin/build-twisted-documentation".  If it's more than that then something is probably wrong...)

> - Beginner's tutorial

... again, tutorial for what?

> - better glossary

Wow, do we even have a glossary?  It seems to me this task might be better served by links to the API docs.

> - move a bunch of stuff out of the Trac wiki, and into the Sphinx project.  There's stuff there that has become de-facto documentation, which really needs to be version-controlled, etc.

+1

> - install docs
> 
> 
> So after looking over Tom's "skeleton" Sphinx project and a night's sleep, I think we're pretty close to on the same page (or at least pages in the same book).  It looks like a lot of this would be covered in Tom's Task-based docs and the Base Documentation section ("Suiting Up", "Diving in").

Great.

> So here's what I'm kind of thinking as far as combining efforts:
> 1) I'll continue with the "Project Documentation" conversion, while Tom works on the other bits.  Should be fairly easy to combine them, I'm thinking.

Getting Tim to help out with some conversion stuff (and you helping with some of his stuff, as well) might accelerate the review process, which has been part of the bottleneck.

> 2) Let's leave the "Project Documentation" pretty much as-is for the moment, until the Sphinx convo is done.
> 3) I wonder if at least some of the "task-based docs" shouldn't be put into the project sections, and then just linked to from the task-based section?
> 
> Thoughts, Tom?
> 
> [As an aside, is there any way to get the Combinator stuff from the old Trac wiki back online, or at least readable someplace? (Hrmmm, it looks like Google's cache has it for the moment).  Also, it would be much easier to get new contributors, especially for Windows, if Combinator worked out of the box on Windows.  There's (or were) 3 or 4 patches in the Divmod SVN repo that needed to be applied in order to get it to work, and now that the Divmod site is offline it's really hard to set it up on a new machine, even for me, and I have a working example to go from.  Yeah it's off topic, sue me.]

The divmod migration is underway, but we're all busy (life, etc) so it's going slowly.  I have a bunch of tasks on my plate that are part of that which I haven't gotten to yet.

Kevin & Tom, you guys should probably stay away from that time sink and keep forging ahead with this great documentation effort, but if anyone else would like to help out we're on #divmod; drop by, say hi, and maybe we can come up with something for you to do...

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20110121/9cbc8c7c/attachment-0001.htm 


More information about the Twisted-Python mailing list