Opened 4 years ago

Closed 3 years ago

#4567 enhancement closed fixed (fixed)

Improve lore2sphinx buildbot results for `projects/core/development`

Reported by: khorn Owned by: lvh
Priority: normal Milestone: Lore to Sphinx
Component: core Keywords: documentation
Cc: thijs Branch: branches/improve-lore2sphinx-output-for-projects-core-development-4567
(diff, github, buildbot, log)
Author: screwtape Launchpad Bug:

Description

Make the lore2sphinx buildbot results for projects/core/development look as good as possible by modifying the lore sources.

Things to look for:

  1. nested inline markup should be removed
  2. whitespace should be added or removed as needed to make ReST markup render properly
  3. anything else that you can improve by modifying the Lore sources

Note that you probably want to run lore2sphinx on a local copy for development purposes.

You can get lore2sphinx from:
http://bitbucket.org/khorn/lore2sphinx

The lore2sphinx buildbot results can be found at:

http://buildbot.twistedmatrix.com/builds/sphinx-html-15615/

(or at different directories for different revisions...the containing directory is browseable)

Attachments (1)

lore-core-development-cleanups.diff (18.5 KB) - added by Screwtape 4 years ago.
Changes that make the lore2sphinx conversion cleaner.

Download all attachments as: .zip

Change History (19)

comment:1 Changed 4 years ago by Screwtape

  • Owner changed from glyph to Screwtape

Changed 4 years ago by Screwtape

Changes that make the lore2sphinx conversion cleaner.

comment:2 Changed 4 years ago by Screwtape

  • Keywords documentation review added
  • Owner Screwtape deleted

Changes made in the attached patch:

  • A couple of instances of "@" have been edited to prevent Sphinx assuming they're email addresses.
  • It turns out that the string "test_" (including the double quotes) causes Sphinx to create a hyperlink to nowhere; we now use <code> instead of <q>, which is arguably more correct.
  • "**kwargs" is marked as code for consistency with the previous mention.
  • The reference to __iadd__ is marked as code, because other Python identifiers are so marked.
  • Various lines had trailing or interior whitespace added to avoid confusing Sphinx.
  • The literal markup/example output sections of doc-standard.xhtml were given captions to help people distinguish between examples and the actual body of the document; <blockquote> elements were also added for the same reason.
  • A link to "coding-standard.xhtml#commits" had the anchor removed because it confused something into making a broken link to "coding-standard-commits".
  • A <code> element was removed from inside an <a> element, since nested markup is not supported.

Changes not in the attached patch:

  • There are still frequent unwanted spaces between commas, colons, full-stops and other punctuation.

comment:3 Changed 4 years ago by exarkun

  • Owner set to exarkun
  • Status changed from new to assigned

comment:4 follow-up: Changed 4 years ago by exarkun

  • Keywords review removed
  • Owner changed from exarkun to Screwtape
  • Status changed from assigned to new
  1. doc/core/development/policy/doc-standard.xhtml
    1. Changes look fine. The lore output is even better than it was.
    2. I noticed a problem with the Sphinx output of "shell" class listings though. They don't appear to be marked up in any way. Is this a general problem with lore2sphinx or the stylesheet that we need to correct?
  2. doc/core/development/policy/coding-standard.xhtml looks good
  3. doc/core/development/policy/svn-dev.xhtml
    1. There's still a ``twisted/protocols/imap4.py in the output
    2. Missing space: appropriate”test-case-name”
  4. doc/core/development/policy/test-standard.xhtml
    1. I suppose eventually we can have a stdlib module role which is both fixed-width like code and a link. Does that make sense? Can you file a ticket for this?
  5. This will also need a news fragment.

comment:5 Changed 4 years ago by screwtape

  • Author set to screwtape
  • Branch set to branches/improve-lore2sphinx-output-for-projects-core-development-4567

(In [29790]) Create branch improve-lore2sphinx-output-for-projects-core-development-4567

comment:6 Changed 4 years ago by screwtape

(In [29794]) Applied patch lore-core-development-cleanups.diff from #4567

refs #4567

comment:7 Changed 4 years ago by screwtape

(In [29795]) Address exarkun's code-review comments.

refs #4567

comment:8 in reply to: ↑ 4 ; follow-up: Changed 4 years ago by Screwtape

  • Keywords review added
  • Owner Screwtape deleted

Replying to exarkun:

I noticed a problem with the Sphinx output of "shell" class listings though. They don't appear to be marked up in any way. Is this a general problem with lore2sphinx or the stylesheet that we need to correct?

I dug into this a little, and it seems that lore2sphinx is rendering all three classes ("python", "python-interpreter", "shell") as the ReST equivalent of HTML's <pre> - unadorned, with no class information. However, Sphinx is smart enough to recognise "python" and "python-interpreter" content and syntax-highlight it anyway. Sadly, this cleverness does not extend to shell.

Apparently if lore2sphinx is updated to produce markup like this:

.. code-block:: python
   print "hello world"

.. code-block:: pycon
   >>> 1+1
   2

.. code-block:: console
   $ twistd --version

...then the Python examples should look the same, but the shell example should at least be highlighted.

Filed #4583.

There's still a ``twisted/protocols/imap4.py in the output

Fixed.

Missing space: appropriate”test-case-name”

Fixed.

I suppose eventually we can have a stdlib module role which is both fixed-width like code and a link. Does that make sense? Can you file a ticket for this?

Filed #4582.

  1. This will also need a news fragment.

comment:9 Changed 4 years ago by Screwtape

Generated Sphinx docs here.

comment:10 in reply to: ↑ 8 Changed 4 years ago by khorn

Replying to Screwtape:

Replying to exarkun:

I noticed a problem with the Sphinx output of "shell" class listings though. They don't appear to be marked up in any way. Is this a general problem with lore2sphinx or the stylesheet that we need to correct?

I dug into this a little, and it seems that lore2sphinx is rendering all three classes ("python", "python-interpreter", "shell") as the ReST equivalent of HTML's <pre> - unadorned, with no class information. However, Sphinx is smart enough to recognise "python" and "python-interpreter" content and syntax-highlight it anyway. Sadly, this cleverness does not extend to shell.

Apparently if lore2sphinx is updated to produce markup like this:

.. code-block:: python
   print "hello world"

.. code-block:: pycon
   >>> 1+1
   2

.. code-block:: console
   $ twistd --version

...then the Python examples should look the same, but the shell example should at least be highlighted.

Filed #4583.

FYI, I've made the changes to do this in lore2sphinx.

comment:11 Changed 4 years ago by khorn

  • Milestone set to Lore to Sphinx

comment:12 Changed 4 years ago by thijs

  • Cc thijs added
  • Keywords review removed
  • Owner set to Screwtape

clients howto:

  • the last sentence of the 2nd paragraph of the Overview section, it ends with Protocol .. That space before the period should go.
  • in the last paragraph of the 'Protocol' section there's a space that needs to go in: Protocol -specific
  • the ircLogBot example has a '../../words/examples/ircLogBot.py' reference, can we get rid of this or turn it into a working link?
  • in the 'Persistent Data in the Factory' section there's a bunch of spaces in the last sentence: '... as self.factory ... case of LogBot , it ...'
  • same issue with the last section of the page, api references have an extra space.

comment:13 Changed 4 years ago by Screwtape

  • Keywords review added
  • Owner Screwtape deleted

Actually, there's spaces after every instance of inline formatting, whether it be a link or code or emphasis. It's apparently one of the pitfalls of computer-generated ReST markup that spacing is difficult to get right, and khorn decided to err on the side of 'too much' space rather than 'too little. I mentioned "There are still frequent unwanted spaces" when I put the changes up for review in comment 2; I probably should have repeated that the second time, sorry.

The bare path above included code-listings (such as ircLogBot.py) bugs me too, but it seems to be an artifact of the lore2sphinx conversion rather than something that we can affect by editing the Lore source. Filed as #4621.

And, uh, now that I think about it, the "clients howto" is part of #4568, not part of the changes in this branch. But thank you for your time, anyway. :)

comment:14 follow-up: Changed 4 years ago by jml

  • Keywords review removed
  • Owner set to Screwtape

In the generated Sphinx documentation:

  • Naming Conventions needs a space in "peer:dataReceived(data)" after the colon.
  • Spurious space in "the documentation standard ," in "Twisted Writing Standard"
  • The "use this template" bit in the "Modules" section of the "Twisted Coding Standard" documentation looks a bit wonky. In particular, the name of the file is just floating above the code listing.

The generated Lore documentation looks fine.

Thanks,
jml

comment:15 in reply to: ↑ 14 Changed 3 years ago by khorn

Replying to jml:

  • Spurious space in "the documentation standard ," in "Twisted Writing Standard"

This isn't fixable by modifying Lore sources. It'll need to be fixed up manually.

  • The "use this template" bit in the "Modules" section of the "Twisted Coding Standard" documentation looks a bit wonky. In particular, the name of the file is just floating above the code listing.

see #4621

The missing space in the naming Conventinos doc does need to be fixed, though.

comment:16 Changed 3 years ago by khorn

  • Owner changed from Screwtape to lvh

OK, none of the remaining issues can be fixed here, so merge this and we'll fix it manually post-conversion.

comment:17 Changed 3 years ago by khorn

Manual fixes to come under #4943.

comment:18 Changed 3 years ago by lvh

  • Resolution set to fixed
  • Status changed from new to closed

(In [31010]) Improved lore2sphinx buildbot results for projects/core/development

Author: Screwtape
Reviewer: khorn, exarkun, thijs, jml
Fixes: #4567

Moving towards Sphinx faster! Yay. Some problems are unfixable, so merging.

Note: See TracTickets for help on using tickets.