[Twisted-Python] Backward incompatible fixes to backward incompatible changes

[Glyph]

On Thu, Apr 25, 2019, at 9:52 AM, Jean-Paul Calderone wrote:
> Hello all,
> 
> I'd like to discuss the project policy/process for dealing with incorrect backward incompatibilities that end up released.
> 
> As a case study, we could look at https://twistedmatrix.com/trac/ticket/9410 <https://twistedmatrix.com/trac/ticket/9410#comment:17> / https://github.com/twisted/twisted/pull/1106 <https://github.com/twisted/twisted/pull/1106>
> 
> In Twisted 16.3.0 the behavior of Request.write was changed so that it raises an AttributeError if called after the connection has closed.  Prior to that release, it silently did nothing in that case.
> 
> Now, three years later, we have a PR which proposed restoring the behavior of silently ignoring the write to a closed connection.
> 
> The original change was incompatible.  The new change is incompatible.  What should win out?

There are two things to address here, I think:

In the general case, can we reverse changes to behavior which were desirable, but by-policy incompatible, and did not go through the incompatibility exception process, without going through the incompatibility exception process?

I'm choosing my words carefully here; changes might be "desirable" even if they're not intentional; they might be something that the caller expects even if they're not explicitly documented.  In general I'd say that we have to be careful with this kind of reversal and we should probably go through the compat exception process.  But I think the change in question doesn't actually fall into this category, because it's an exception that nobody wanted, wasn't documented, and wasn't intentional.  So it's in this other category:

Should we consider the raising of an undocumented exception to be a compatibility surface we must maintain compatibility with?

In this case, I think it depends on the type of the exception - specifically, whether the exception is defined in Twisted itself, or defined in a library that is clearly being invoked and whose behavior should be propagated.  So, for example, if you call connection.foo().bar() and you get FooCannotBeBarredError, this might something to be considered a compatibility concern.  However, if you call connection.foo().bar() and get an undocumented stdlib exception (KeyError, AttributeError, ValueError), I think it's okay to implicitly treat these as "gross violations of specifications".

In the specific case you dealt with I agree with your evaluation and I'm glad we've landed the bugfix :-).

-g
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20190430/9ea1c31e/attachment-0002.html>