[Twisted-Python] [RFC] Deprecation Policy
glyph at divmod.com
glyph at divmod.com
Mon Apr 28 02:50:59 EDT 2008
On 18 Apr, 05:06 am, jml at mumak.net wrote:
>Now that the release is out, I'd like to start work on fixing #1216
>-- official deprecation policy.
Sorry for the delay. Due to a miscommunication, I thought that Chris
would be the one writing up our meeting notes and announcing this.
After I realized I was the one who was supposed to be doing it, I wanted
to write up something official-sounding on the wiki first, but of
course, there's never enough time. So here's a rough cut at the policy
that we (the current "TSF board", such as it is: chris, me, JP, and
itamar) agreed on at the sprint in Cambridge.
Thanks, Jonathan, for motivating this; we've needed a clear statement
for a long time. Someone (and this will probably be me) needs to make a
complete and unambiguous index of official Twisted policy and put it
together somewhere, probably on the wiki, at least at first.
>I think these are all the questions we need to answer:
The proposed policy does break down as a list of answers to these
questions, so one point at a time:
>- Is it time-based or release-based?
Both. There will be a minimum amount of time and a minimum number of
releases before a deprecated feature may be removed.
>- How long for?
Every deprecation must last for a minimum of 2 releases and a minimum of
one year. Of course, anyone who wants to keep maintaining deprecated
APIs for longer than that may do so.
I know this is somewhat longer than you've suggested, but at the meeting
we ran down a laundry list of real-world usages of Twisted and pretty
much nobody upgrades Twisted more than once per year. In fact, once per
2 years seems to be the average, and that's a fairly ambitious estimate.
The objective of providing a year-long deprecation period is to allow a
user who is performing their bi-yearly Twisted upgrade to do so with no
more than 2 intermediary upgrades.
>- Do we grade deprecations? How?
There will be three phases of deprecation. When a deprecation is first
introduced (in trunk) the "deprecation counter" will be at 3. When that
deprecation sees its first release, it decrements to 2. Deprecations at
levels 3 and 2 will be PendingDeprecationWarning. When that same
deprecation sees its second release after a 6-month period, the counter
decrements to 1, and finally, after a second 6-month period, it
decrements to 0 and can be removed. Deprecations with a counter at 1
and at 0 (if they still exist) should emit a DeprecationWarning.
The counter may not move down past 2 until *all instances of the emitted
warning* have been removed from the Twisted test suite, with the
exception of tests specifically for the deprecated functionality itself
(i.e. tests that call assertWarns).
The reason that this is described as a counter rather than primarily in
terms of (Pending)DeprecationWarning is that this was a subject of
intense debate during the meeting. There is definitely a consensus that
the Python warnings system is not a sufficient tool we have to deal with
the kind of notifications that we want to provide. There is also a
consensus that it is the only tool available. So, let me provide some
more detail on the desired behavior here.
The idea behind using PendingDeprecationWarning is that *users* of a
piece of software which depends on Twisted should, in principle, be able
to upgrade Twisted by one version without seeing any error output.
However, *maintainers* of that software should immediately see all the
warnings when they test with a new release. Trial should change to show
all PendingDeprecationWarnings by default. However, this might be
impractical due to other uses of PendingDeprecationWarning - if that is
the case, Twisted should include its own warning class. It may be
helpful to do that in order to include the additional metadata required
for an automated deprecation workflow, however. The desired deprecation
* Twisted developer deprecates functionality, by adding a
warnings.warn(TwistedDeprecationWarning(some useful metadata)) to the
* The release process automatically runs "update-deprecations" over the
codebase, updating the deprecation counters in the developer-supplied
metadata appropriately. The release goes out. (Ideally this would be
done without actually having to modify the whole codebase, updating a
central deprecation / release index.)
* Another release happens that sets some counters to zero.
* A developer drops support for some deprecated functionality,
confident in their ability to do so by observing the zero (or negative)
value for the counter.
An obvious remaining piece of work to implement this final policy is to
define some temporary manual things that we can do until such a tool is
implemented, and implement the tool itself as soon as possible. I don't
believe any tickets have yet been filed for this, and *that* I am
leaving in our venerable release manager's court, since this is all a
part of the release process except for the fairly minor change to trial.
However, in the absence of any tools, this policy does apply now, it
will just be a bit of a pain to work out where the counter stands on any
particular deprecation until we've got some more metadata to help us.
>- Which code is subject to this policy?
All Python code released in Twisted, with the exceptions of private
modules and test modules.
Only code which has been present in a release needs to be deprecated.
Anything which was added after the last release and has not yet been
present in any other release may be modified at will. (We try to keep
trunk stable, but this only applies to code that was relying on features
in previous releases.) I think that CompatibilityPolicy's explanation
of an "incompatible change" explains this.
Users: do not import anything from a package named "test", or any name
(module, class, or function) which begins with "_"; bears will eat you.
This code may change at any time.
(Not a part of the policy: I personally want to add some caveats here,
where users of Twisted can ask for a grace period on private APIs that
people actually needed to use, but I think for now let's err on the side
of allowing the deprecation - I don't want to encourage people to use
private APIs. Users: if you find that there are things you can't do
with the public APIs and you really need a grace period to upgrade your
applications, please speak up. Eventually, I'd like to have a very
aggressive "you can't even import (or invoke) this if it's private"
regime where users can't accidentally use private functionality without
first seeing a runtime error of some kind that they have to explicitly
silence, as with my "pyexport" hack. Once we've done *that* we should
definitely have a zero-tolerance policy on supporting private stuff,
There's obviously some work to do here to make this policy easy and
convenient to follow. This needs to be edited, CompatibilityPolicy
updated, and prominently linked. We need someone to write up at least
two different summaries: one for maintainers of projects using Twisted
("What kind of compatibility should I expect from Twisted releases?")
one for Twisted contributors ("How do I add a deprecation? When can I
remove it?") and maybe one for users and one for packagers, too. We
need some tools to help us update deprecations. We need trial to change
so that developers are notified earlier than their users.
Do I hear any volunteers...? :)
More information about the Twisted-Python