[Twisted-Python] glossary (was Re: Using SerialPort with t.a.s.Application)

[Kevin Horn]

On Fri, Jan 28, 2011 at 7:50 PM, Glyph Lefkowitz <glyph at twistedmatrix.com>wrote:

>
> On Jan 28, 2011, at 5:47 PM, Kevin Horn wrote:
>
>
>
> On Fri, Jan 28, 2011 at 1:33 PM, Glyph Lefkowitz <glyph at twistedmatrix.com>wrote:
>
>>
>> On Jan 28, 2011, at 10:38 AM, Kevin Horn wrote:
>>
>>
>>
>> On Fri, Jan 28, 2011 at 7:25 AM, Itamar Turner-Trauring <
>> itamar at itamarst.org> wrote:
>>
>>>
>>> A service is supposed
>>> to be something you can start and stop, and encapsulates a
>>> self-contained piece of business logic.
>>>
>>> -Itamar
>>>
>>>
>>>
>> This or something very much like it should be in the Twisted Glossary.
>>
>> Kevin Horn
>>
>>
>> The whole idea of a glossary concerns me a little bit.
>>
>
> You're just saying that because you don't want anyone else to understand
> what you're saying ;)
>
>
> I am aggrieved that you would insinuate any premeditated obfuscatory animus
> appertaining to my person or those of my associates.  I maintain an
> amaranthine focus on limpid exposition and efficacious intercommunication.
>
>
Hee!  Told you so :)

> Seriously though.  When I first started thinking about ways to improve the
> documentation.  The _first_ thing I thought was missing was decent
> explanations of some of these things.  Whether it's in a glossary or
> someplace else, clear definitions need to exist for all the major Twisted
> concepts (people may disagree on what is or is not major, and that's fine).
> Putting these definitions (or links to them) in a glossary keeps them all
> together and easy to find and refer to.  One of the first thing I look for
> in any software documentation is a glossary or some set of definitions that
> I can refer to while I'm _reading_ the API docs or whatever.
>
>
> If you need a glossary open while you're reading the API docs, that means
> that the docs you're reading are insufficiently marked up.  The word
> 'service' (in this context) should _always_ itself be a hyperlink to the
> IService interface.  We do this pretty consistently in all new
> documentation.  Similarly 'deferred' should always be a link to
> twisted.internet.defer.Deferred.  <
> http://epydoc.sourceforge.net/epytext.html> has more detail on how to
> format these if you want to do clever stuff like have a lower-case word like
> 'service' link to the IService docs.
>
>
You don't need the glossary open while reading the API docs, you need it
open while reading the other stuff.  Hopefully the need for this will fade
as the organization of the existing long-form docs improve.

Plus, you don't necessarily need it "open".  You can mark up a glossary term
in Sphinx and it'll automatically create cross-refs, so you say "What the
heck does that mean?", click the link, and it takes you to the term on the
glossary page.  From there you can get to the API docs, sure.  There should
absolutely be a link there.  But there should also be something that tells
you whether you need to dive into the API docs.


> I'm totally on board with making a *list* of these common abstractions
> outside the API docs, but 99% of what that list should do is provide a
> definitive index of links into the API docs to call out the top important
> interfaces you need to read about.
>

See.  Right there.  You assumed the reader would be looking for what
interfaces they need to read about.  If I'm new to Twisted, and I see some
example code on the web someplace, I'm not going to _see_ the interface it
implements.  I'm going to see probably the class.  So I Google it.  Assuming
I know anything about interfaces (which might be assuming a bit, but might
not), I'll basically know what that is, but right now the API docs aren't a
lot of help for older code, and most of the core things are older code.  And
that assumes there is an interface for whatever abstraction I'm looking
for.  I spent several hours once upon a time sifting through the API docs
and source trying to figure out what (Cred) Avatars came with Twisted.  Some
abstractions don't come with source code.

Now these assumptions are certainly valid in many, if not most, cases.  But
keep in mind we want to make things easy for the new people too.  That's how
we get more new people.


> Also, the existing API docs for many of the core concepts are terrible, or
> even (cringe) non-existent in places, so they could certainly stand to be
> improved.  I'm not saying our existing docs are great, just that we should
> have one great index of terms and not five mediocre ones.
>

Yep.  I have a list of docs tickets to go through, but there's millions more
missing (OK, hopefully not millions, but...).

One of them (at least) is to fix the current glossary, since it's current
state is DISMAL (It has links to the API docs everywhere though).


> Solving problems through software requires finding the right abstractions,
> which IMO depends a great deal on finding the right vocabulary for those
> abstractions.  I know the Twisted devs spend a lot of time and energy trying
> to find exactly the right word for a class, or a method, or whatever.  Let's
> communicate that to new people!  If you don't want a "glossary", then we'll
> do something else, but I think it's important to have all of the really
> critical concepts described in one place somewhere.
>
>
> I should have qualified my discomfort with the term 'glossary' a bit more.
>  We do need the thing you're talking about.  My only concern is redundancy.
>  Even if we had awesome API documentation for all of these things, there's
> still a bootstrapping phase where people are not sure which classes are
> one-off utilities and which things are core abstractions they'll be using
> every day.
>

Yes, exactly.

> Also consider the problem of reverse human lookup:  "Oh man, what was that
> thing called that stops and starts Twisted code at the appropriate times?"
>
> If you stick that into a search engine, you probably won't find it for a
> loooong time.  But if there's a glossary, you just look there, and you find
> it pretty easily.
>
>
> Well, that exact phrase, no; but "start and stop twisted application", and
> "start and stop twisted service" both point at the right API doc.  We should
> probably work a bit harder to make 'start stop twisted' find that page first
> :).
>

OK, well when I ran it through Google the API page didn't even show up on
the first page.  The application framework page did.  Google farm oddness
maybe.


> One way I frequently see it done is vague little snippets of text like
>> this, and I don't like that.
>>
>
> Well, I consider it much more meaningful than what is currently there.  Not
> saying it's really sufficient, but it is succinct.  I think what I'd like to
> see is a "glossary" entry with:
>
> - an API link (e.g. "An object which implements the IService Interface" --
> and yes, it should be a proper English sentence.)
>
>
> This I certainly agree with.
>
> - A succinct definition (like Itamar's quip above)
> - a paragraph or two of discussion (not too long though!)
>
>
> Why can't this just be in the API doc?
>

Well, it can and should, but:
- not every abstraction and/or thing we want to define has source code.
- some people are intimidated by having to go to API docs just to find a
little information about a library.  The way I see it, people go to the API
docs when they want to _use_ a library.  They go to the long-form docs when
they want to find out if library X is the one they want to use.  Perhaps we
disagree here.
- I foresee circumstances where the API docs might not be available.  (e.g.
If you have no net access and you don't have PyDoctor installed...which I
doubt many people do.)  But you might have Sphinx installed.  Or have the
handy-dandy Windows compiled HTML help file for Twisted (which doesn't exist
yet, but should someday...maybe even soon).  Sure you can look in the source
code, but it's kind of a pain if all you want is a definition.

After the Sphinx conversion it _might_ be possible to pull in docstrings
from the API docs into the Sphinx docs.  It certainly would be if the API
docs were marked up according to Sphinx conventions, but even I'm not crazy
enough (yet) to try to change over all of that stuff.  I might be crazy
enough to try to write a piece that converts the epydoc markup into Sphinx
on the fly and then processes it into html or whatever, but I might also
come down with a sudden severe case of sanity.


> - "see also" links to HOWTO's etc.
>
>
> Again - why not in the API doc?  If someone were to find the concept via
> some other documentation that described the term in question as a return
> type or argument rather than a generic word, presumably that person might
> also want to read the narrative documentation.
>
>
It is currently not possible (AFAIK) to link to things in the long-form docs
from the API docs in any kind of maintainable way.  There's a ticket for
this (#2801).  I have some ideas for how it might be fixed after the Sphinx
conversion is done.  I have zero idea how it might be done before that.  You
could put in static hyperlinks in the API markup, but that would be highly
subject to breakage.


> I'm not sure how much of a problem maintaining it would be, especially as I
> agree that linking to the API docs is probably a good idea (though I think a
> lot of the docstrings in there could be greatly improved).  For one thing,
> how often has the _intent_ of the IProtocol interface(which has no
> docstring, btw) changed?
>
>
> So, first: aaaaaauuugh.  Is there a doc bug for that?  That is an interface
> which really, really needs a docstring.  Can you submit a patch, like, right
> now? :)
>
> Second: the intent hasn't changed often.  But clearly we need to update the
> documentation to document that intent better, and I hope that happens more
> often than it does.  Narrative vs. reference documentation is already a
> split, albeit a necessary one in my opinion.  Narrative vs. code-reference
> vs. language-reference seems like we're just splitting hairs.
>
>
But the "language-reference" is an adjunct to the narrative documentation,
IMO.  Like an index, or the table of contents. And when I go to it, I don't
expect/want there to _only_ be an API link there

> And how often does someone show up on IRC who doesn't understand what a
> Protocol object is show up?
>
>
> Actually... I don't remember IProtocol being a big problem.  Which is
> slightly surprising, given that it has no docstring.  Maybe everyone who has
> run into a confusion at this basic stage has just given up, or maybe the
> method docstrings are good enough, or maybe the learn about it from a
> tutorial.
>
>
It seems to me like there's at least one person a week on IRC who needs the
old talk about what Protocols do vs. what Factories do.  Or how they get
their Protocol to make multiple connections.

[Aside: docstring for t.i.p.Factory: "This is a factory which produces
protocols. By default, buildProtocol will create a protocol of the class
given in self.protocol.".
docstring for t.i.i.ProtocolFactory: "Interface for protocol factories."
Aaaaaaaugh!  Some of the old docstrings need a lot of work. It's a separate
issue though.]

Thoughts?
>>
>>
> I think we've totally hijacked this thread :)
>
>
> Yes, retitled appropriately.
>
>
Well done.

In the words of Inigo Montoya, "Lemme sum up":

- I don't think we're too far away from each other in our opinions, but I
think you're more worried about redundancy than I am.  I think once the docs
are more navigable, the glossary will be much more useful, and problems will
be noticed much more quickly than they are now.

- If you want a great example of how I think a glossary should be used with
Sphinx, check out the Pyramid docs (
http://docs.pylonsproject.org/projects/pyramid/dev/index.html). Like that
but with more API links. :)

- In addition, I also think there should be a "Twisted Concepts"
document/section in or near the hopefully-soon-to-be-written intro stuff
that Tom is working on.

- it's going to be an incremental process.  I'm sure we'll have plenty of
time for discussion. :)

Kevin Horn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20110128/85813baa/attachment.html>