[Twisted-web] Nevow and python 3

[Glyph]

On Wed, Jan 24, 2018, at 12:54 AM, Markus Demleitner wrote:
> Hi Glyph,
> 
> On Tue, Jan 23, 2018 at 11:51:44PM -0800, Glyph wrote:
> > > On Jan 23, 2018, at 2:03 AM, Markus Demleitner <msdemlei at ari.uni-heidelberg.de> wrote:
> > > through from twisted.web quite unsatisfying.  I've tried to formulate
> > > something like a preliminary policy in
> > > https://github.com/msdemlei/nevow/blob/master/doc/strings_and_bytes.rst,
> > > but I'm deeply unhappy about most of this.
> > 
> > Yeah, IRequest is an unfortunate interface.  Work is ongoing to
> > design a more thoughtful one and gradually migrate to it.  I think
> > https://github.com/wsanchez/txrequest
> > <https://github.com/wsanchez/txrequest> is getting pretty complete,
> > and from there the hope is that it will get integrated into Klein
> > and then upstreamed into Twisted.
> 
> ...which of course looks like a very long migration path, very likely
> stretching far beyond python2's EOL if my typical deployment is on
> Debian stable...

Shorter the more people we have working on it :-).

> I wonder if one couldn't arrange for inevow.IRequest to adapt to
> essentially this (perhaps with a bit more glue code to keep
> as compatible with existing code as possible).

This would be great.

> > > Ideally, I'd like to fairly exclusively expose actual strings to
> > 
> > "actual" meaning text?
> 
> Yes: str.
> 
> > > (2) Is anyone at all interested in the livepages stuff (I'm not, and
> > > dropping it would certainly make porting and maintaining the package
> > > a lot easier)?
> > 
> > If you're going to drop livepage, is there any reason you're not
> > just migrating to twisted.web.template?  Is there any way t.w.t
> > could be made into an easier migration path?
> 
> I have to admit I've not really investigated t.w.t in any detail; one
> thing that actually does matter -- because templates of my users will
> silently fail with it -- is the change of the XML namespace for the
> nevow elements and attributes.  
> 
> I suppose one could work around it on template load, though.  Hm --
> this might be an interesting alternative to keeping all of nevow.

I don't pretend to have a complete solution here - unfortunately I've ported very few projects *from* nevow *to* t.w.t so almost no work has gone into migration or compatibility.  But I do think there's definitely something to investigate.

> Of course, that still doesn't help against the insiduous breakage
> when suddenly code needs to deal with byte strings coming out of
> request.args.keys(), path components, and headers; that's a bit less
> of a direct concern for me since it's much less deployer-visible for
> my applications, but if I have to write some compatibility layer, why
> not have nevow be it?

This is largely about aligning as much effort as possible in one place :).  Our goals are not exactly the same of course, you want Nevow+Py3, and most Klein interest right now is primarily around APIs rather than HTML, but this is exactly why extracting the request-processing fix to the lowest shared level is important :).

> > > (3) If I went ahead with this and actually tried to get a
> > > python3-nevow into Debian (which I'd really, really like to see): How
> > > is everyone's feeling about keeping the name nevow even if serious
> > > functional changes (such as dropping livepages) were to come about?
> > > [What I care about most is keeping the templating system including
> > > the render_ and data_ methods].
> > 
> > 
> > If you're going to do a fork, please come up with a new name.
> > Using the existing one is confusing, it causes namespace conflicts
> > (what happens when Nevow itself gets some upstream patches that
> > port it to python 3 from someone else?) and it causes import
> > conflicts.
> 
> Fair enough (a bit more on this in my response to Werner).

I do hope we can figure out a way to work together! Although if a fork is necessary there will definitely be no hard feelings - the logistics on that are hard.

> > That said, if you are interested in porting Nevow the same way that
> > Twisted is being ported (i.e. there's one set of sources, but `pip
> > install` on python 3 only installs the modules ported thus far)
> 
> Ouch -- given my experiences while porting it would seem a major
> chore to keep python2 compatibity.  But then I've not tried, and I
> may be wrong.   If I actually wanted to follow
> https://twistedmatrix.com/trac/wiki/Plan/Python3 by the letter,
> that's certainly far beyond what I can reasonably commit in terms of
> resources.

Doing this at the very start of the project (Python2.5 + Python3.2) was agonizing; barely practical. But today (Python2.7 + Python3.4) is mostly quite pleasant.

> > then that would certainly be a way to invest the effort in just
> > porting the parts that you need, and letting someone else port
> > LivePage later :).  I'd be happy to make you a maintainer if you're
> > interested in doing this - not a lot of other activity on Nevow
> > right now!
> 
> I'd certainly like to make the effort I invest as worthwhile as
> possible; but, in particular after the experience of porting, I feel
> I'm not nearly as familiar with nevow's codebase (and customs in the
> twisted ecosystem) as I should be to confidently work as maintainer.

There's really only one way to become familiar with these things - jump right in! :)

> Before I commit to anything, let me try to work out if I can get
> nevow to the point where it doesn't cause extra porting pains in the
> python3 migration of dependent code.

I don't want to pressure you into any commitments you're uncomfortable with; as you said, just wanting to making the investment of your effort worthwhile.  Sounds like your mind is in the right place though; let me know when you have completed your investigations :).

-g