[Twisted-Python] Library Versioning
James Y Knight
foom at fuhm.net
Tue Apr 20 19:22:12 EDT 2004
Yeah, everyone's probably going to get mad at me for bringing
versioning up again, but here goes.
I'll structure the first bit as an argument between me and some
argumentative type who is unconvinced.
Me: I want to be able to have two apps installed that use different,
incompatible revisions of twisted or twisted subprojects.
Other: You can just install multiple versions of Twisted, in different
directories, and set your PYTHONPATH differently for the different
apps. And this works *right now*.
Me: Yes, that is true. But I want a solution to work for e.g.
distributed debian packages. If twisted core makes an incompatible API
change and becomes major revision 2, I want to be able to install
applications that depend on v1 and applications that depend on v2 at
the same time. I want to be able to apt-get install AppA which uses on
libtwisted-core-1, and AppB which uses libtwisted-core-2 and have them
Other: Okay, so call the newer API twisted_2 and the old one twisted_1
(or just twisted). What's the problem?
Me: A few problems. First problem is you have to litter the entire
codebase of an application with the version numbers. Every time you
import the module you have to add the revision. When you want update
your app to use 2.0, you need to change every single file to use
twisted_2, even if they otherwise needed no changes. What I really want
to do is put the required version in *ONE* place in the app. Perhaps in
its __init__.py, or (if it's an app), perhaps in its executable script
Other: No, you *DON'T* want that, you really do want to specify the
version in every import because you want to be able to import both
versions in one "app".
Me: But it is unlikely that two revisions that don't know about
eachother will both work in one app, so while it may look like I can
import twisted_1 and twisted_2, mixing them will likely cause my app to
Other: Right, which is why the new version supplies the old interface
as well. So the Twisted 2.0 library can provide both twisted_2 *AND*
twisted_1 reimplemented in terms of twisted_2. And then you *can* use
both in one app and everything will be happy.
Me: Yes, that is possible. If you do that, you're not breaking API
compatibility, and thus the new version isn't actually a major revision
at all. I do concede, it is _always_ better to not break API
compatibility if you can avoid it. But I only know of one major
opensource library that is making a guarantee to never release a new
major version: glibc. Most projects do not have unlimited programmer
time, and keeping backwards compatibility forever can take a lot of
time and testing. Also note the "again" there.
Me: I think that having every API version supported by one library
version is a very different thing than just being able to install
multiple distinct versions of the library. Even if the goal is to
support every API revision for eternity, it's probably not always
possible, and at that point, you would want multiple library versions
Thus, I propose the following magic (I'll have you take for granted
that might work, for the moment):
This explicitly *DOES NOT* attempt to solve the multiple APIs supported
by one installed library version issue. That is a separate issue.
- "import twisted; twisted.setversion(xxx)" loads version xxx of the
twisted library. If another version has already been loaded, or if the
specified version isn't available, raise an exception.
- "import twisted" uses the already loaded version if one is loaded, or
else looks for the latest installed version and loads that if none have
After that, you can use twisted.whatever normally. The setversion only
needs to be done once per app/library e.g. in the top level
__init__.py. It can be done in more places if you want.
This has the following desirable characteristics to me:
1) Simple case for beginners -- they don't have to do anything about
versioning at all. Just import twisted normally and they get the latest
version they have installed (which may very well also be the only
version they have installed).
2) Yet, still easy to add the version requirement to an app that didn't
have it initially. That you make sure to call setversion only really
matters for packaged or widely distributed apps/libs. So if you're
package an app that didn't have a setversion call initially, add it --
a one line patch.
3) Allows distro packaging to work sensibly. Incompatible revisions can
be installed side-by-side and apps that use the old one can continue
working just fine.
4) Doesn't require possibly unfulfillable compatibility promises from
developers going forward.
This kind of versioning is somewhat similar to that of Python itself:
- You have a python app, and you can specify the version required on
the first line of the main program (#!/usr/bin/python2.3), or else not
- Multiple versions are simultaneously installable.
- New versions have some compatibility, but don't promise perfect
compatibility. Thus, sometimes you don't need to do anything to make
your app work with the new version. But, if you do, there's likely only
a few things you need to change, not every file.
- If the new version of python implements a compatibility API this is a
separate issue from the python version: to get the old API you do
nothing, and to get the new API you do: from __future__ import
PS: (The magic above does seem to be implementable, see PMW, Python
More information about the Twisted-Python