[Twisted-Python] Lore and generating reStructuredText (Lore2Sphinx)

[Kevin Horn]

On Mon, Mar 25, 2013 at 6:29 PM, Glyph <glyph at twistedmatrix.com> wrote:

>
> On Mar 25, 2013, at 9:16 AM, Kevin Horn <kevin.horn at gmail.com> wrote:
>
> How can you tell that this was created by a directive?  You can't, because
> it could just as easily have been a list full of links to begin with.
>
>
> But, I don't care if it was created by a directive or not.
>
> I think we're talking about two different things.
>
> What you seem to be talking about is using Sphinx to do source-to-source
> Lore-to-ReST transformation.  In that case, you're (sort of) right, in that
> information is lost when you invoke directives.  If we did this, and it
> worked, it would just be a slightly better way to implement lore2sphinx;
> we'd still need to manage the transition in largely the same way.
>
> What *I'm* talking about is just using Lore source as an input to Sphinx,
> and going straight to the output HTML.  In order to do this, we just need
> to construct the right tree and actually *invoke* the directive callables
> at the right time.  They produce whatever output they want to produce, and
> we hand that back to Sphinx, and it outputs some docs.  With this strategy,
> we just switch to sphinx by switching our build process; we don't switch
> input formats.  Then, if someone wants to use Lore they can, if they want
> to use ReST they can, and we can migrate on an as-needed basis; there's no
> need for a single big format migration for us to start using Sphinx.
>
>
Hmmm.  We are indeed talking about two different things.

What you describe is probably technically possible, but I still don't think
it's a very good approach.  It seems to me that it would be very brittle
and error prone.  You'd need to:
- figure out the node output of every directive you were trying to
replicate, with every type of (tedious in the best case, possibly very
tricky for some directives, but maybe not too bad)
- figure out a system to "inject" those nodes at the right time (not sure
how difficult this would be)
- figure out how Sphinx modifies the doctrees to do all it's linking, index
generation, toctree handling, etc. (I think this is the hardest part)
- figure out a way to get Sphinx to take doctrees as input (I'm guessing
you'd serialize them in whatever way Sphinx does when it does it's caching,
stick them in the cache, and then get Sphinx to build from the cache?)

What about changing the build process to use some kind of tool that goes
over the doc files, and if a source file is in lore format, it translates
it using rstgen, and if it's already a rst file, it just copies it into the
Sphinx project as-is?  Then just build the Sphinx project.  I think this
would be a lot less hassle than trying to decipher and replicate a bunch of
docutils and Sphinx internals, and would really only be a minor change to
the way that the lore2sphinx command line tool already works.  It just
processes each lore file and sends the output to an output directory, so
you'd just need to modify it to skip the processing and only copy rst files.

With the new refactoring of lore2sphinx into lore2sphinx-ng, I think this
is possible and would yield acceptable results.  Of course someone still
would need to finish lore2sphinx-ng and rstgen, but that's either going to
have to happen anyway, or some other tool would have to be built that mucks
with doctrees.

Thoughts?

--
Kevin Horn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20130326/46223bda/attachment.htm