<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Oct 25, 2015, at 5:48 PM, Amber Hawkie Brown <<a href="mailto:hawkowl@atleastfornow.net" class="">hawkowl@atleastfornow.net</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Hi everyone,<br class=""><br class="">As many know, one of the things that makes the Twisted project so unique is our conformance to our Compatibility Policy. This policy means that users of Twisted can freely upgrade between versions with all incompatibilities being warned about before causing code to break. However, for a while, one part of the compatibility policy has caused issues - what we do with code that has been deprecated for a long time, and at least the release + 2 & 1 year after.<br class=""><br class="">The existing policy states:<br class=""><br class="">"Removal should happen once the deprecated API becomes an additional maintenance burden. For example, if it makes implementation of a new feature more difficult, if it makes documentation of non-deprecated APIs more confusing, or if its unit tests become an undue burden on the continuous integration system. Removal should not be undertaken just to follow a timeline."<br class=""><br class="">This makes the only reasonable cause for any code being removed from Twisted is if it is a maintenance burden, but does not take into account the effect that keeping large amounts of deprecated code has on new, existing, and future Twisted users. It does specify that it should be removed if it makes documentation of non-deprecated APIs confusing, but by its very existence, it makes what should be "best practice" more confusing -- especially if the deprecated API is "simpler" at first glance, but was deprecated because of vast underlying issues,<br class=""><br class="">Discussions have come to the conclusion that the exact reverse of this policy should be instated:<br class=""><br class="">"Removal of code should occur as soon as the deprecation grace period has ended."<br class=""><br class="">The reason for this is that a leaner Twisted is a better Twisted -- we should strive to not break existing applications, but we have the deprecation policy in place to ensure that breakage is, if all goes to plan, never out of the blue. Less code surface means that Twisted is easier to learn for new users and Twisted-using projects onboarding new people, easier to use for established users with a clear best practice, and easier to maintain because the codebase is not a web of things we deprecated and then never removed. We believe this change benefits everyone.<br class=""><br class="">This is also similar to the deprecation policy of another time-based releasing project, Django (<a href="https://docs.djangoproject.com/en/1.8/internals/release-process/#internal-release-deprecation-policy" class="">https://docs.djangoproject.com/en/1.8/internals/release-process/#internal-release-deprecation-policy</a>), where releases are every 9 months and code is *always* removed in Release where deprecated + 2, giving them a deprecation grace period of up to 1 and a half years.<br class=""><br class="">If you have any opinions on this change, please let me know, and we'll take it all into account before changing the policy.<br class=""></div></div></blockquote></div><br class=""><div class="">I am generally in favor, for the reasons that the other participants already said: removing things faster results in a cleaner codebase that's easier to figure out.</div><div class=""><br class=""></div><div class="">However, I feel like making "get rid of deprecated code faster" be the <i class="">first</i> thing we do to mitigate these costs might be putting the cart before the horse. The costs of maintaining large piles of legacy API surface are definitely not trivial. A few are:</div><div class=""><br class=""></div><div class=""><ol class="MailOutline"><li class="">Maintainers have to read and understand more code when maintaining a feature, regardless of whether that code ultimately impacts the code being modified.</li><li class="">Testing more code means longer test runs for everyone. Older, deprecated code is more likely to have test cases that require the reactor to run, so in many cases it can also mean less reliable test runs.</li><li class="">Example code, in Twisted, in Twisted's documentation, and on other sites, may continue to refer to deprecated APIs. Given that this <i class="">appears</i> to work, the transition takes longer because <i class="">new</i> users start using old APIs.</li><li class="">Twisted's users, both new and old, have to refer to API documentation that describes numerous different ways to do something, and deprecations are not clear.</li></ol><div class=""><br class=""></div></div><div class="">For problems 1 and 2, there's nothing much we can do but delete the code itself. However, for problems 3 and 4, there are two other solutions which I think that we should explore as well, and we should possibly try to get in place <i class="">before</i> more aggressively deleting the actual code.</div><div class=""><br class=""></div><div class="">For one thing, we currently do not fail our tests on, or even ratchet on, warnings. If I turn on warnings (meaning "export PYTHONWARNINGS='all::DeprecationWarning'"), purely at test _startup_ I see these:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class="">/Users/glyph/Projects/Twisted/twisted/internet/endpoints.py:29: DeprecationWarning: twisted.internet.interfaces.IStreamClientEndpointStringParser was deprecated in Twisted 14.0.0: This interface has been superseded by IStreamClientEndpointStringParserWithReactor.<br class=""> from twisted.internet.interfaces import (<br class="">/Users/glyph/Projects/Twisted/twisted/spread/jelly.py:92: DeprecationWarning: the sets module is deprecated<br class=""> import sets as _sets<br class="">/Users/glyph/Projects/Twisted/twisted/scripts/test/test_tap2deb.py:8: DeprecationWarning: tap2deb is deprecated since Twisted 15.2.<br class=""> from twisted.scripts import tap2deb<br class="">/Users/glyph/Projects/Twisted/twisted/scripts/test/test_tap2rpm.py:15: DeprecationWarning: tap2rpm is deprecated since Twisted 15.2.<br class=""> from twisted.scripts import tap2rpm<br class="">/Users/glyph/Projects/Twisted/twisted/words/test/test_msn.py:28: DeprecationWarning: twisted.words.protocols.msn was deprecated in Twisted 15.1.0: MSN has shutdown.<br class=""> from twisted.words.protocols import msn<br class=""></div></blockquote><div class=""><br class=""></div><div class="">not to mention the scads of additional warnings that our test suite spits out over the course of its run. We really need to get rid of these warnings to be sure that it's even <i class="">possible</i> to actually remove the deprecated APIs, and to ensure that there's not something internal that they're really necessary for.</div><div class=""><br class=""></div><div class="">For another, we have no way to <i class="">hide</i> deprecated APIs in the API documentation. If a new user is trying to figure out how to do something, if they're doing it during the deprecation period, they should not start out by using a deprecated API; yet, sometimes, if you look at the docs, that's what you'll find first.</div><div class=""><br class=""></div><div class="">Also, we haven't done a serious overhaul of our documentation in far too long, and we have no automated tooling to tell us if it's producing warnings or even tracebacks.</div><div class=""><br class=""></div><div class="">So, while I think we should start removing more deprecated APIs, I think it's important to address these deficiencies <i class="">first</i>, so that when we perform those removals we have a higher degree of confidence that we're not breaking our own documentation and breaking programs that users wrote last week and thought were working fine because they didn't notice the fine-print "deprecated" at the end of a docstring. If we <i class="">can't</i> figure out how to get our own house in order in terms of keeping up with deprecations, it seems like we are likely to create unforseen problems for our users.</div><div class=""><br class=""></div><div class="">-glyph</div><div class=""><br class=""></div></body></html>