Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#4939 enhancement closed fixed (fixed)

twisted.web lacks a built-in templating engine

Reported by: habnabit Owned by: glyph
Priority: normal Milestone: Twisted-11.0
Component: web Keywords:
Cc: jknight, itamar@… Branch: branches/web-template-4939
(diff, github, buildbot, log)
Author: habnabit Launchpad Bug:

Description (last modified by glyph)

There's no templating engine for rendering things like error pages and directory listings.

I'm going to port over nevow.page.Element into twisted.web.template.

The goal for this should be to support the serialization idiom of HTML5, rather than XHTML. The implementation should not have to change much beyond the DOCTYPE declaraitons and xmlns attributes to support that though, as I understand it.

Eventually supporting XHTML (is it XHTML5 now?) should be possible as well since it's a couple of trivial serialization tweaks to the same DOM, but small bugs will be coped with better by an HTML5 parser (a little extra whitespace instead of a PARSE ERROR page in the browser).

Attachments (1)

locale-independent-formatting-with-tests-r3.patch (5.8 KB) - added by djfroofy 4 years ago.

Download all attachments as: .zip

Change History (65)

comment:1 Changed 4 years ago by DefaultCC Plugin

  • Cc jknight added

comment:2 Changed 4 years ago by habnabit

  • Author set to habnabit
  • Branch set to branches/web-template-4939

(In [30993]) Branching to 'web-template-4939'

comment:3 Changed 4 years ago by habnabit

  • Milestone set to Twisted-11.0

comment:4 Changed 4 years ago by habnabit

  • Branch changed from branches/web-template-4939 to branches/webhtml-4939

(In [31041]) Branching to 'webhtml-4939'

comment:5 Changed 4 years ago by itamar

  • Cc itamar@… added

James had some good questions:

  1. Any reason you're not just fixing the few XSS attacks in twisted?
  2. Does Twisted really need *more* code?

A separate templating library that is nevow without cruft seems like a fine idea, but why does it need to be in twisted?

comment:6 Changed 4 years ago by jknight

I'm not even sure nevow without cruft needs to exist; there's about 5000 templating libraries already in existence, Twisted should recommend that people use any one they like (perhaps out of some list that is known not to do terrible things).

comment:7 Changed 4 years ago by habnabit

Responding to just jknight for now: there's no other templating engine that I've seen which has as complete of separation between code and markup.

comment:8 Changed 4 years ago by habnabit

  • Status changed from new to assigned

comment:9 Changed 4 years ago by habnabit

  • Keywords review added
  • Owner habnabit deleted
  • Status changed from assigned to new

comment:10 Changed 4 years ago by habnabit

  • Branch changed from branches/webhtml-4939 to branches/web-template-4939

comment:12 Changed 4 years ago by djfroofy

Ignore the above patch which was supposed to be for 4937. Sorry.

comment:13 Changed 4 years ago by habnabit

(In [31139]) Removing code, adding tests.

There was a lot of dead, unused, or otherwise terrible code which I've pruned
out again. Some things were replaced with better alternatives, which had
corresponding unit tests added. Coverage is all-around better, which is between
87% and 92% across all of the new code. Most of the untested code is in
repr methods and other such things.

I still need to write docstrings for a bunch of things, including the new
tests. Also, docs. I at least understand everything better to be able to write
new documentation for the code.

refs #4939

comment:14 Changed 4 years ago by habnabit

(In [31140]) Renaming 'invisible' to 'bucket'.

refs #4939

comment:15 Changed 4 years ago by habnabit

Also re-forced a build; new results should appear soon.

comment:16 Changed 4 years ago by habnabit

(In [31218]) Refactoring code, adding docs.

  • Refactoring preprocessors to be exposed in the same way that renderers are.
  • Unifing add-child and add-attribute syntax in _stan to both use call.
  • Adding in untested code for integration with twisted.web.server.
  • Adding in some skeletal-ish documentation for using twisted.web.template; it needs a lot of work, but this at least will get it some exposure.

refs #4939

comment:17 Changed 4 years ago by glyph

  • Description modified (diff)

Added comment about desired version.

I think valid XML is valid HTML5, is that right?

comment:18 Changed 4 years ago by glyph

  • Owner set to glyph
  • Status changed from new to assigned

comment:19 follow-up: Changed 4 years ago by lvh

I think valid XML is valid HTML5, is that right?

Um, not quite.

Valid X*HT*ML is pretty close to being valid HTML5. You would need to fix the DOCTYPE, and potentially change some <meta> elements in <head>, though. It is true that closed tags like <br /> are valid HTML5 (but the SGML form is preferred).

comment:20 in reply to: ↑ 19 Changed 4 years ago by glyph

Replying to lvh:

I think valid XML is valid HTML5, is that right?

Um, not quite.

Right, yeah.

Valid X*HT*ML

OK, yes, I realize that you can't have arbitrary tags in valid HTML these days, they have to be part of the spec. Also: no xmlns attributes, the namespaces are implciit in the serialization.

is pretty close to being valid HTML5. You would need to fix the DOCTYPE, and potentially change some <meta> elements in <head>, though.

I do know about <!DOCTYPE html>, so let's do that. But, what <meta> and <head> elements?

It is true that closed tags like <br /> are valid HTML5 (but the SGML form is preferred).

Yeah, but it's valid, and the serializer can be simple if we just do it the same way in both.

comment:21 Changed 4 years ago by glyph

  • Keywords review removed

Thanks again. I think this is actually getting close. I knew there were
coverage issues, but I was pleasantly surprised to discover that only one or two
of them were actually functional, almost all of them were just __repr__s or
no-ops of some kind. (Does 'coverage' even emit stuff for 'pass'? Maybe I
shouldn't have bothered commenting on those, because they should just be
docstrings.)

  1. Lots of things need docstrings. Especially everything in the tests; twisted.test.test_flatten doesn't have any docstrings at all. I'm not going to enumerate all of these in a list, because it's kinda tedious and it's easy to see the things that don't have docstrings. A method with a 'pass' is a dead giveaway, especially :).
  2. According to coverage run trial twisted.web.test.test_flatten twisted.web.test.test_stan twisted.web.test.test_template, there are several bits of code that haven't been covered by the added unit tests. I'll enumerate them here as a hopefully helpful laundry list for anyone responding to the review:
    1. twisted.web.template - many of these methods do nothing, but if they need to be implemented at all, then surely that null implementation is there to do something. It should be tested.
      1. _ToStan.resolveEntity
      2. _ToStan.skippedEntity
      3. _ToStan.processingInstruction
      4. _ToStan.startElementNS
        1. the if nons in attributes suite - line 144
        2. the negative branch of if nsPrefix is None - line 153
        3. Both AssertionErrors - lines 158 and 162.
      5. startDTD
      6. endDTD
      7. comment - the ignoreComment early-return.
      8. flattenToRequest. No docstring and zero coverage for this function, but its name implies it's pretty darn important.
    2. twisted.web._element (You can really tell that this is the new, good code that was developed TDD: 97% coverage.)
      1. _Expose.__call__: the TypeError
      2. _Expose.get: the UnexposedMethodError
    3. twisted.web._flatten
      1. FlattenerError._formatRoot - completely uncovered.
      2. FlattenerError.__repr__
      3. FlattenerError.__str__ - but this can just be deleted, as __repr__ will automatically delegate.
      4. UnfilledSlot.__str__
      5. UnsupportedType.__str__
    4. twisted.web._stan
      1. directive.__hash__
      2. directive.__cmp__
      3. slot.__iter__
      4. Tag.__iter__.
        1. Also this uses old exception syntax, don't do that.
      5. Tag._clearSpecials
      6. Tag._clone, the first two 'if' branches
      7. UnsetClass.__nonzero__
      8. CDATA.__repr__
      9. Comment.__repr__
    5. In the tests, there's some uncovered code as well. This indicates that maybe not all of the test cases were brought over and some of the other coverage problems would go away if it were.
      1. twisted.web.test.test_flatten.TestSerialization._test_cancellingSerialization is never called.
  3. Several names do not conform to the coding standard. no_ns_attrs, for example. Some of these are just copying the names used in nevow, but this is an opportunity to make them compliant. I suggest attrsWithoutNS. And flattenstring should be flattenString. I see you already fixed XMLFile though: thanks!
  4. It looks like we're not going to do the sphinx migration before the release, so the docs need to be translated to lore. I plan to do that by hand unless someone has a great idea for how it might be done more quickly.
  5. docFactory has an unfortunate name. The method is called 'load', the role is referred to as a 'template loader'. The classes are named 'something file' or 'something string'. The class attribte is docFactory; but what is it a factory of, and what do docs have to do with it? I'm thinking that it should be called something like loader = or template = in the class, but I haven't fully decided yet. This doesn't really need to be dealt with, but it would be a bonus if it could be.

I think I'm going to deal with most of this feedback myself, perhaps
coordinating with others for parts of it. Hopefully we can get it reviewed by
tomorrow.

comment:22 Changed 4 years ago by habnabit

(In [31249]) Adding docstrings for twisted.web.test._util.

refs #4939

comment:23 Changed 4 years ago by habnabit

(In [31250]) Adding docstrings for twisted.web.test.test_flatten.

refs #4939

comment:24 Changed 4 years ago by mithrandi

comment:25 Changed 4 years ago by glyph

(In [31256]) Not the way I'd implement this, but lucky for me there are no tests, so I'll just delete it. 100% coverage in twisted.web.template.

refs #4939

comment:26 Changed 4 years ago by glyph

As of r31264 we are up to 100% coverage in all the new code, as far as I can tell.

comment:27 Changed 4 years ago by jerub

(In [31265]) Add docstring for 'renderFactory' arg to _flatten, rename _flattensome to
_writeFlattenedData.

refs #4939

comment:28 Changed 4 years ago by jerub

(In [31266]) Remove inAttribute arg from deferflatten()

Looking at inAttiribute, it's specifically for dealing with <foo
bar="bar&quot;bar"> quoting, i.e. stopping XSS attacks when a slot contains
data that needs to be quoted differently in attribute context.

When passing something more complicated than a string or something that calls
back with a string, such as a Tag instance: inAttribute is reset and quoting
doesn't work in a way that is useful.

Removing inAttribute from the public api makes the code's external interface
slightly easier to understand.

comment:29 Changed 4 years ago by jerub

(In [31267]) Rename deferflatten and flattenstring to flatten and flattenString respectively.

refs #4939

comment:30 Changed 4 years ago by jerub

(In [31268]) Add docstrings to test_template.py.

Also add an extra test for multiple preprocessors sent to load()

refs #4939

comment:31 follow-up: Changed 4 years ago by jknight

Why not just remove inAttribute entirely, and always escape "? Makes the internal API simpler too.

comment:32 Changed 4 years ago by jerub

(In [31297]) Taupe and Purple!

Move all decorators to being @decorators because we don't need
to support python2.3

refs #4939

comment:33 Changed 4 years ago by jerub

(In [31298]) Rename _iterflatten and _flatten to _flattenTree and _flattenElement respectively.

refs #4939

comment:34 in reply to: ↑ 31 ; follow-up: Changed 4 years ago by glyph

Replying to jknight:

Why not just remove inAttribute entirely, and always escape "? Makes the internal API simpler too.

I think the answer has something to do with JavaScript. If you could definitively demonstrate that you can put &quot; in a script tag and have JavaScript think that it's really a '"', then great, let's do that.

comment:35 in reply to: ↑ 34 Changed 4 years ago by mithrandi

Replying to glyph:

Replying to jknight:

Why not just remove inAttribute entirely, and always escape "? Makes the internal API simpler too.

I think the answer has something to do with JavaScript. If you could definitively demonstrate that you can put &quot; in a script tag and have JavaScript think that it's really a '"', then great, let's do that.

JavaScript in XML/XHTML is basically a disaster anyway. You can't have a literal "<", unless you wrap it in a CDATA section, but then HTML parsers can't handle it anymore (and nobody is actually serving XHTML as application/xhtml+xml, so you're always dealing with an HTML parser). CSS has the same problem, although "<" occurs less frequently in CSS. You almost always want to put CSS and JavaScript in external resources.

comment:36 Changed 4 years ago by glyph

  • Keywords review added
  • Owner glyph deleted
  • Status changed from assigned to new

Dear Reviewer,

You will find recent test results here, some excellent narrative documentation for your perusal here, and documentation generated with pydoctor here. For your convenience, please also find a full diff of the changes included in the branch here.

I sincerely hope this all meets with your approval.

Faithfully yours,

-glyph

comment:37 Changed 4 years ago by glyph

Most Magnanimous Reviewer,

Here's an correct link to the pydoctor/API documentation (whoops!), as well as a test coverage report, generated by running "coverage run trial twisted.web.test.test_flatten twisted.web.test.test_stan twisted.web.test.test_template". Please sort by coverage descending (click on the 'coverage' column twice) and note that all the new modules (_stan, template, _element, _flatten) all have 100% coverage. (I removed a little bit of dead code, and thus the buildbot tests have been re-run as well.)

Thank you,

-glyph

comment:38 follow-up: Changed 4 years ago by jknight

I think the answer has something to do with JavaScript. If you could definitively demonstrate that you can put &quot; in a script tag and have JavaScript think that it's really a '"', then great, let's do that.

You can't. A <script> tag is kind of an implicit CDATA section in HTML (although with special rules of its own). You can't use *any* HTML escapes in it, including &lt;. And literal '<' characters are usually fine to include literally, unless they start with "<script" or "</script". And, "of course", you can't emit an HTML comment in a script state and expect it to actually be treated as commented out...

Double-quotes aren't a *special* escaping problem in the script context.

And don't forget about the rest of the special nodes too, including at least style and textarea...

comment:39 in reply to: ↑ 38 Changed 4 years ago by glyph

Replying to jknight:

I think the answer has something to do with JavaScript. If you could definitively demonstrate that you can put &quot; in a script tag and have JavaScript think that it's really a '"', then great, let's do that.

You can't. A <script> tag is kind of an implicit CDATA section in HTML (although with special rules of its own). You can't use *any* HTML escapes in it, including &lt;. And literal '<' characters are usually fine to include literally, unless they start with "<script" or "</script". And, "of course", you can't emit an HTML comment in a script state and expect it to actually be treated as commented out...

Double-quotes aren't a *special* escaping problem in the script context.

And don't forget about the rest of the special nodes too, including at least style and textarea...

Ugh. I had happily forgotten most of this stuff, but it comes creeping back...

Maybe XML serialization is a good idea. Really strict XHTML parsing lets you do all of this junk with the normal XML rules, right?

comment:40 Changed 4 years ago by mithrandi

  • Keywords review removed
  • Owner set to glyph
  1. twisted.web._element
    1. _Expose.__call__: The docstring refers to Expose instead of _Expose
    2. Element.renderer: The name of this method feels really confusing to me, given how many other things are already named renderer. How about renaming it to getRenderer?
  2. twisted.web._flatten
    1. flattenString: Single quotes are used everywhere in this function, with one inexplicable exception - the only string constants that contain double quotes are themselves written using double quotes, requiring backslash-escaping.
  3. twisted.web._stan
    1. allowSingleton
      1. The HTML spec refers to these elements as "void elements", so I think voidElements (or something to that effect) would be a better name for this list.
      2. I called it a list, but it's actually a tuple; stylistically, I would prefer a real list, but I don't know actually know what the Twisted coding standard is regarding list vs tuple.
      3. command, embed, keygen, source, track, and wbs are missing from the list.
    2. slot: The name, children, and default attributes are not documented.
    3. Tag
      1. The docstring refers to "Prototype", with a capital "P". I think this was meant to refer to Proto, which no longer exists, so this description needs to be fixed up a bit.
      2. slotData is undocumented.
    4. Tag.__call__: The special if not kw path seems superfluous.
  4. I'd just like to say that this branch is SUPER FRIKKIN' AWESOME!!!111oneone

So uh yeah awesome. Everything in my list is pretty straight-forward, so go ahead and merge once everything has been dealt with, unless further discussion seems warranted.

comment:41 Changed 4 years ago by itamar

Aren't you still in the middle of a discussion about how the serialization is problematic?

comment:42 follow-up: Changed 4 years ago by mithrandi

Replying to itamar:

Aren't you still in the middle of a discussion about how the serialization is problematic?

I believe the current serialization is fine; the discussion is basically about a proposed simplification of the serialization, which could still be implemented after this branch lands.

Replying to glyph:

Ugh. I had happily forgotten most of this stuff, but it comes creeping back...

Maybe XML serialization is a good idea. Really strict XHTML parsing lets you do all of this junk with the normal XML rules, right?

If you are actually using XHTML, then yeah, all of the normal XML rules apply, and you don't have to worry about all the crazy HTML rules. There are several problems with this, though:

  1. XHTML parsing rules are just XML parsing rules, so no error recovery is allowed; this is sometimes good, but sometimes you don't want an ugly disturbing XML parser error when something goes wrong.
  2. Using XHTML requires you to serve your content with a content-type of application/xhtml+xml, which will completely prevent your page from being displayed in older browsers.
  3. Most(?) browsers do not support incremental rendering (and parsing?) of XHTML content.

As a result, "nobody" actually serves up application/xhtml+xml, they all serve up text/html, and thus HTML parsing rules apply, even if you're cheating and using XML/XHTML processing tools to generate your content.

comment:43 Changed 4 years ago by glyph

(In [31347]) Expose is already in a private namespace. No need to make it double-secret. refs #4939 review point 1.1

comment:44 Changed 4 years ago by glyph

(In [31348]) .renderer(name) -> .lookupRenderMethod(name). We already have more than enough stuff called 'renderer'. refs #4939 review point 1.2

comment:45 Changed 4 years ago by glyph

(In [31349]) adjust peculiar quoting. refs #4939 review point 2.1

comment:46 Changed 4 years ago by glyph

(In [31350]) allowSingleton -> voidElements; plus, a documentation reference which explains that the spec calls them that too. refs #4939

comment:47 Changed 4 years ago by glyph

(In [31351]) Document name, children, default attributes. (Also: goodbye slots, no tests fail, write a benchmark if somebody cares about the performance of slot().) refs #4939 review point 3.2

comment:48 Changed 4 years ago by glyph

(In [31352]) get hip to the latest standards. refs #4939 review point 3.1.3

comment:49 Changed 4 years ago by glyph

(In [31353]) Remove reference to "Prototype", generally clean up "Tag" docstring. refs #4939 review point 3.3.1

comment:50 Changed 4 years ago by glyph

(In [31354]) Document slotData. refs #4939 review point 3.3.2

comment:51 Changed 4 years ago by glyph

(In [31355]) All things being equal, less code is better. And there is no benchmark proving that earlying out of an empty iteritems() call actually helps. refs #4939 review point 3.4

comment:52 follow-up: Changed 4 years ago by jknight

I believe the current serialization is fine; the discussion is basically about a proposed simplification of the serialization, which could still be implemented after this branch lands.

Well, my original comment was a simplification. glyph brought up that actually the current serialization basically doesn't work at all for script/style content...which necessitates more complication being added.

comment:53 in reply to: ↑ 52 Changed 4 years ago by glyph

Replying to jknight:

I believe the current serialization is fine; the discussion is basically about a proposed simplification of the serialization, which could still be implemented after this branch lands.

Well, my original comment was a simplification. glyph brought up that actually the current serialization basically doesn't work at all for script/style content...which necessitates more complication being added.

I think it's a good point to be aware of, and we should definitely improve this later, but as I recall, correct cross-browser parsing behavior inside arbitrary <script> tags was a complete nightmare. It's one thing to try to avoid XSS and cleanly emit text when putting content into the middle of a page; it's quite another to try to allow taking advantage of the most obscure horrible corners of browser parsing.

As you rightly pointed out, the only right way to do this is to treat any javascript or CSS beyond onclick="foo()" or style="font-size: big" as an external resource, not something quoted inline in your page.

I also agree that we should probably rip out inAttribute completely at some point, it's a purely internal detail for right now, but the current code is nice and well tested and that simplification could easily happen later.

comment:54 Changed 4 years ago by jknight

html5lib has the following list of tags for which the content is implicit raw data, and thus shouldn't be escaped (html5lib.constants.rcdataElements): 'style', 'script', 'xmp', 'iframe', 'noembed', 'noframes', 'noscript'.

The simplest correct thing would be to just not quote them on output. And then also explain to people that they need to quote the content of those tags on *input* because twisted.web.templates uses XML not HTML. And then also just tell users to not use "&lt;/$tagname" in those contexts, because that just can't be serialized properly.

title and textarea are slightly different in that escapes are accepted, but even if you don't escape, it's still part of the text, not separate elements. It's impossible to include other elements inside one of those contexts. That is, in:
<textarea><!-- Comment --><h1></textarea>

The "comment" is not a comment, it is part of the text. The <h1> "element" is not an element, it's also part of the text. But it's fine to continue quoting text on output for those two.

comment:55 follow-up: Changed 4 years ago by jknight

As you rightly pointed out, the only right way to do this is to treat any javascript or CSS beyond onclick="foo()" or style="font-size: big" as an external resource, not something quoted inline in your page.

That's not true at all: it's trivial to correctly include complex JS or CSS inline in your page, as long as you know whether you're outputting HTML or XHTML. The nightmare scenario occurred in Nevow because it was (or maybe just the poor users were) never sure if it was reading and outputting html or xhtml, and so you had to try to be compatible with both in the same document...

comment:56 Changed 4 years ago by glyph

Replying to jknight:

html5lib has the following list of tags for which the content is implicit raw data, and thus shouldn't be escaped (html5lib.constants.rcdataElements): 'style', 'script', 'xmp', 'iframe', 'noembed', 'noframes', 'noscript'.

Good to know. None of these tags seems like the kind of thing you should be putting user input into, so I think we're okay :).

We should have a separate ticket for making these work right though, insofar as it's possible.

The simplest correct thing would be to just not quote them on output. And then also explain to people that they need to quote the content of those tags on *input* because twisted.web.templates uses XML not HTML. And then also just tell users to not use "&lt;/$tagname" in those contexts, because that just can't be serialized properly.

Yes, this is my recollection as well. There are some strings that you just can't convince an HTML parser to preserve in any sane way, so you have to raise exceptions during rendering if they're present.

title and textarea are slightly different in that escapes are accepted, but even if you don't escape, it's still part of the text, not separate elements. It's impossible to include other elements inside one of those contexts. That is, in:
<textarea><!-- Comment --><h1></textarea>

... but <textarea> &lt; </textarea> will still be seen as a text area with "<" in it, so our current algorithm is in good shape, yes?

The "comment" is not a comment, it is part of the text. The <h1> "element" is not an element, it's also part of the text. But it's fine to continue quoting text on output for those two.

Thanks for the input on this.

comment:57 in reply to: ↑ 55 Changed 4 years ago by glyph

Replying to jknight:

As you rightly pointed out, the only right way to do this is to treat any javascript or CSS beyond onclick="foo()" or style="font-size: big" as an external resource, not something quoted inline in your page.

That's not true at all: it's trivial to correctly include complex JS or CSS inline in your page, as long as you know whether you're outputting HTML or XHTML. The nightmare scenario occurred in Nevow because it was (or maybe just the poor users were) never sure if it was reading and outputting html or xhtml, and so you had to try to be compatible with both in the same document...

If you're outputting XHTML, it can trivially be made to work.

If you're outputting HTML though... I'm not so sure. I can't remember the specific issues, but remember that "HTML" implicitly includes IE6, which means that all the rules are sometimes randomly thrown in the trash. I am fairly certain that there are byte sequences which do seriously different things on IE6 and IE7 as compared to every other browser, and some older versions of firefox used to trip over themselves as well (although nobody really uses an old version of firefox, so we can stop caring about that).

I'm sure that anything even halfway modern will work with the HTML quoting rules you've described, though. And my intention would be to add the "are you serializing HTML or XML, let's be completely clear" API to deal with this issue.

I think I should probably add something to the 'doctype' section of the narrative docs that makes it clear that <script> and <style> tags aren't fully supported yet.

comment:58 Changed 4 years ago by mithrandi

Replying to jknight:

html5lib has the following list of tags for which the content is implicit raw data, and thus shouldn't be escaped (html5lib.constants.rcdataElements): 'style', 'script', 'xmp', 'iframe', 'noembed', 'noframes', 'noscript'.

Blegh. According to the WHATWG spec, <style> and <script> (the other elements don't exist) are "raw text", not RCDATA. But maybe this is just bad naming in the html5lib code? <title> and <textarea> *are* RCDATA, but aren't mentioned in your list.

(The difference is that RCDATA can have character references, but no ambiguous ampersands; raw text cannot have character references, and thus ambiguous ampersands are allowed.)

comment:59 Changed 4 years ago by glyph

(In [31356]) address issues raised by jknight - explain to the user that if they are going to play with a syntax flamethrower they should put on some asbestos ampersands first

refs #4939

comment:60 follow-up: Changed 4 years ago by glyph

jknight: I think we can table the issue, but it would not be good to let users fall into this booby trap completely unawares.

I added a note to the section on formats and doctypes explaining the limitations of not caring whether you're emitting HTML or XML. If you take the appropriate care to set up everything so that it's XML, it currently works; we should have a bug to make it work in XML. In the meanwhile though, users should take care to avoid doing templating in those types of tags.

comment:61 in reply to: ↑ 60 ; follow-up: Changed 4 years ago by jknight

Replying to glyph:

jknight: I think we can table the issue, but it would not be good to let users fall into this booby trap completely unawares.

Okay.

I added a note [...]

Your note makes it sound like there's only a problem if you include dynamic content in it, but really you can't output a script at all, even if it's static in the template file. I also think that XHTML output should never be recommended for anyone to use, because it doesn't work well.

comment:62 Changed 4 years ago by glyph

  • Resolution set to fixed
  • Status changed from new to closed

(In [31357]) Merge web-template-4939 - incorporate the good parts of Nevow into Twisted.

Author: habnabit, glyph, lewq, jesstess, jerub, exarkun

Reviewer: glyph, idnar

Fixes: #4939

This change pulls in exarkun's latest and greatest page rendering model (hence
his well-deserved inclusion on the authors list, even though he didn't make
commits to this particular branch) from nevow.page.Element, with several
modifications to simplify it and reduce the amount of code in it. Also included
is a new narrative tutorial document explaining how to use basic templating, and
also that you DO NOT NEED THIS to generate HTML in a Twisted program; you
just might want it because it's easy and fun.

This should allow us to much more gracefully and correctly generate HTML within
Twisted in the few places we already do, and perhaps also allow us to more
easily provide new browser-based, simple management tools for Twisted servers at
some point in the future.

comment:63 in reply to: ↑ 61 Changed 4 years ago by glyph

Replying to jknight:

Replying to glyph:

jknight: I think we can table the issue, but it would not be good to let users fall into this booby trap completely unawares.

Okay.

I opened #4972 for fixing this for real somehow.

I added a note [...]

Your note makes it sound like there's only a problem if you include dynamic content in it, but really you can't output a script at all, even if it's static in the template file. I also think that XHTML output should never be recommended for anyone to use, because it doesn't work well.

I think my note is clear in the context that it's presented, but things like this could always be clearer. If you want to open a doc bug for a better explanation before #4972 gets addressed, please go ahead.

comment:64 in reply to: ↑ 42 Changed 4 years ago by glyph

Replying to mithrandi:

  1. Most(?) browsers do not support incremental rendering (and parsing?) of XHTML content.

For what it's worth, Wikipedia disagrees, and says this is a fixed problem: http://en.wikipedia.org/wiki/XHTML#cite_note-mature_support-15

Note: See TracTickets for help on using tickets.