[Twisted-Python] Twisted 16.6.0rc1 Release Candidate Announcement

Glyph Lefkowitz glyph at twistedmatrix.com
Tue Nov 22 19:03:07 MST 2016


> On Nov 22, 2016, at 20:27, Mark Williams <markrwilliams at gmail.com> wrote:
> 
> On Tue, Nov 22, 2016 at 06:31:45PM -0500, Glyph Lefkowitz wrote:
>> 
>> OK, this whole time I thought we were talking about a sensible application of text_type to the API, perhaps with some leniency for bytes-ish-ness on python 2.  I haven't reviewed the PR, I was just responding to the concerns as raised on the list.
> 
> Sorry - I didn't mean to steer this towards API bike shedding.
> 
>> If it's just randomly encoding on one version and not the other, and correct usage of the API depends on *users* doing 'if PY2:' in their own code, then perhaps Mark's concern is indeed well-founded and we should roll it back before 16.6.
>> 
> 
> Tristan's exactly right.  Furthermore, if we decide to make IRCClient
> call its various command methods with unicode strings on Python 2,
> we'll be breaking backwards compatibility.  This is what I meant when
> I wrote:
> 
> On Nov 20, 2016, at 19:35, Mark Williams <markrwilliams at gmail.com> wrote:
>> 
>> Yes.  Here's the lede: IRCClient should deal in bytes and we should
>> introduce a ProtocolWrapper-like thing that encodes and decodes
>> command prefixes and parameters.  It should implement an interface,
>> and we can start with an implementation that only knows about UTF-8.
>> The obvious advantage of this is that you can more easily write
>> IRCClients that work on both Python 2 and 3.
>> 
> 
> But it totally wasn't clear - sorry!
> 
> Of course, I also want IRC client implementation that lets me get at
> bytes, but that's a discussion I'll move to a new thread.
> 
> Given the inconsistency between Python 2 and Python 3, do we proceed
> with the revert?

Okay.  So.

The rule for reverts like this is: if you do something today, which is correct usage of the API and produces an observably correct result, will that be broken in the future if we fix it?  If so, then we need to revert because the interface as released is unsupportable.

As it stands, we have a matrix of 4 behaviors:


bytes
text(ascii)
text(nonascii)
py2
works
works
UnicodeDecodeError
py3
garbage
works
works

This... is actually... fine, surprisingly.

The right thing to do is to write code that passes text all the time.  If you do that right now, it'll work on py3 and raise an exception on py2, unless it happens to be ASCII, in which case it'll work.

If you write code that passes bytes on py3, it'll just be garbage.  But, we want to deprecate that anyway, and you can't get correct, usable behavior out of it, no matter what workarounds you stuff in; so it's a bug, and can be fixed like any bug.

Similarly if you pass non-ascii text on py3, you'll get a UnicodeDecodeError.

This is not a good situation, but it's totally fixable without breaking the interface.  We just fix the py2 version to accept text_type as well, and if Mark sneaks in a patch that makes py3 do the right thing with bytes, well, I don't know that I can stop him.

More importantly, it would probably be a smaller change to fix the methods (we could even fix them one at a time; say, action, join, etc) than to un-port and re-port the whole thing.

So: yes, it's broken, and in a worse way than I thought.  To get it to the point where we can actually implement logic consistently between two versions, we need to add a flag to IRCClient's constructor which is default-false on py2 and default-true on py3 which says "give me text", so that callbacks like privmsg and joined can start receiving text_type on py2 as well as py3; right now it has to receive str because they've previously received str.  But that's a separate issue.

I am open to the idea that I have evaluated this incorrectly though, since this has been possibly the most confusing change since https://twistedmatrix.com/trac/ticket/411 <https://twistedmatrix.com/trac/ticket/411>.  But as of right now I still think we shouldn't revert.

-glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20161122/a3233e3b/attachment-0002.html>


More information about the Twisted-Python mailing list