[Twisted-Python] Library Versioning

[James Y Knight]

Yeah, everyone's probably going to get mad at me for bringing 
versioning up again, but here goes.

I'll structure the first bit as an argument between me and some 
argumentative type who is unconvinced.

=========
Me: I want to be able to have two apps installed that use different, 
incompatible revisions of twisted or twisted subprojects.

Other: You can just install multiple versions of Twisted, in different 
directories, and set your PYTHONPATH differently for the different 
apps. And this works *right now*.

Me: Yes, that is true. But I want a solution to work for e.g. 
distributed debian packages. If twisted core makes an incompatible API 
change and becomes major revision 2, I want to be able to install 
applications that depend on v1 and applications that depend on v2 at 
the same time. I want to be able to apt-get install AppA which uses on 
libtwisted-core-1, and AppB which uses libtwisted-core-2 and have them 
both work.

Other: Okay, so call the newer API twisted_2 and the old one twisted_1 
(or just twisted). What's the problem?

Me: A few problems. First problem is you have to litter the entire 
codebase of an application with the version numbers. Every time you 
import the module you have to add the revision. When you want update 
your app to use 2.0, you need to change every single file to use 
twisted_2, even if they otherwise needed no changes. What I really want 
to do is put the required version in *ONE* place in the app. Perhaps in 
its __init__.py, or (if it's an app), perhaps in its executable script 
file.

Other: No, you *DON'T* want that, you really do want to specify the 
version in every import because you want to be able to import both 
versions in one "app".

Me: But it is unlikely that two revisions that don't know about 
eachother will both work in one app, so while it may look like I can 
import twisted_1 and twisted_2, mixing them will likely cause my app to 
blow up.

Other: Right, which is why the new version supplies the old interface 
as well. So the Twisted 2.0 library can provide both twisted_2 *AND* 
twisted_1 reimplemented in terms of twisted_2. And then you *can* use 
both in one app and everything will be happy.

Me: Yes, that is possible. If you do that, you're not breaking API 
compatibility, and thus the new version isn't actually a major revision 
at all. I do concede, it is _always_ better to not break API 
compatibility if you can avoid it. But I only know of one major 
opensource library that is making a guarantee to never release a new 
major version: glibc. Most projects do not have unlimited programmer 
time, and keeping backwards compatibility forever can take a lot of 
time and testing. Also note the "again" there.

Me: I think that having every API version supported by one library 
version is a very different thing than just being able to install 
multiple distinct versions of the library. Even if the goal is to 
support every API revision for eternity, it's probably not always 
possible, and at that point, you would want multiple library versions 
installed simultaneously.



=========

Thus, I propose the following magic (I'll have you take for granted 
that might work, for the moment):

This explicitly *DOES NOT* attempt to solve the multiple APIs supported 
by one installed library version issue. That is a separate issue.

- "import twisted; twisted.setversion(xxx)" loads version xxx of the 
twisted library. If another version has already been loaded, or if the 
specified version isn't available, raise an exception.
- "import twisted" uses the already loaded version if one is loaded, or 
else looks for the latest installed version and loads that if none have 
been loaded.

After that, you can use twisted.whatever normally. The setversion only 
needs to be done once per app/library e.g. in the top level 
__init__.py. It can be done in more places if you want.

This has the following desirable characteristics to me:
1) Simple case for beginners -- they don't have to do anything about 
versioning at all. Just import twisted normally and they get the latest 
version they have installed (which may very well also be the only 
version they have installed).
2) Yet, still easy to add the version requirement to an app that didn't 
have it initially. That you make sure to call setversion only really 
matters for packaged or widely distributed apps/libs. So if you're 
package an app that didn't have a setversion call initially, add it -- 
a one line patch.
3) Allows distro packaging to work sensibly. Incompatible revisions can 
be installed side-by-side and apps that use the old one can continue 
working just fine.
4) Doesn't require possibly unfulfillable compatibility promises from 
developers going forward.


This kind of versioning is somewhat similar to that of Python itself:
- You have a python app, and you can specify the version required on 
the first line of the main program (#!/usr/bin/python2.3), or else not 
(#!/usr/bin/python).
- Multiple versions are simultaneously installable.
- New versions have some compatibility, but don't promise perfect 
compatibility. Thus, sometimes you don't need to do anything to make 
your app work with the new version. But, if you do, there's likely only 
a few things you need to change, not every file.
- If the new version of python implements a compatibility API this is a 
separate issue from the python version: to get the old API you do 
nothing, and to get the new API you do: from __future__ import 
whatever.

James

PS: (The magic above does seem to be implementable, see PMW, Python 
Mega Widgets.)