[Twisted-Python] application release with Twisted included (was: Evangelism notes...)

[Bob Ippolito]

On May 4, 2005, at 5:42 PM, Darran Edmundson wrote:

> Bob Ippolito wrote:
>
>> That's why it's better if you don't install Twisted, just make  
>> its  source code part of your project.  That's what you should be  
>> doing  for just about any dependency, especially those where you  
>> don't need  extension modules.
>>
>
> Could anyone offer advice on the best way to do this?   For example,
> the setup.py of our twisted app creates a package hierarchy:
>     - agdevicecontrol
>         - agdevicecontrol.server
>         - agdevicecontrol.client
>         - agdevicecontrol.gui
>         - agdevicecontrol.common
>         - agdeviceconttrol.devices
>
> This is all under subversion control and (inspired by Twisted) a few
> hours away from having its tests run under Buildbot.  So now if we
> want to include Twisted2.0 and the ZopeInterfaces dependencies
> in our source release, should we just take a source snapshot and
> add:
>         - agdevicecontrol.twisted
>         - agdevicecontrol.zopeinterfaces

Don't try and change the Python module hierarchy.  Just make twisted  
a sibling of your package.  zopeinterfaces is a bit different because  
it *does* have extension module(s), and doesn't evolve very quickly,  
so it might be best to just require that to be installed.  If you  
only support one platform you could install a particular version of  
zope.interfaces with a --prefix that points somewhere in your  
repository (such that it's a sibling of your code) and just check  
that in.  Or you can go with something a bit more complicated with  
multiple platforms where you have a bootstrap script that mangles the  
sys.path so that it finds the appropriate platform specific modules.

This is a "__bootstrap__" module that I've used in the past that I  
have used before:

import os, sys
# change to current dir
curdir = os.path.dirname(os.path.abspath(__file__))
os.chdir(curdir)
# add dependencies and platform specific path for dependencies
sitepkg = os.path.join(curdir, 'site-packages')
uname = os.uname()
platdir = '%s-%s' % (uname[0].replace(' ', '_'), uname[-1].replace('  
', '_'))
sys.path[1:1] = [
     sitepkg,
     os.path.join(sitepkg, platdir),
     os.path.join(sitepkg, platdir, 'PIL'),
]

In this particular case, the layout looked like this:

site-packages/
     formless/
     nevow/
     twisted/
         ... these three are just checkouts of their respective  
projects at some point in time
     plat-Darwin-Power_Macintosh/
         mx/
         PIL/
         sqlite/
         _sqlite.so
     plat-Linux-i686/
         ... same stuff as in Darwin

The way it bootstraps is that I had a "start.sh" script that calls "./ 
twistd -o -y something.tac", where twistd is just a local copy of the  
twistd shell script that is modified to import __bootstrap__ before  
it does anything else.

If you didn't notice, zope.interfaces isn't here.. because this is a  
copy of Twisted 1.3.  nevow worked better that way at the time.  It  
still works just fine, and I don't have to worry about other  
applications on the server getting in the way because the environment  
is pretty isolated.  I can svn co the project on (almost) any Mac OS  
X or Linux machine with a Python 2.3 interpreter and just run it,  
without worrying about any other dependencies.

> And do we try to keep these in synch with the respective
> CVS/SVN repositories?  Or since this is going out to end
> users, stick with stable releases (almost rhetorical question).

That's up to you.  You might want to map external code (even specific  
revisions of external code) as just svn:externals, but since you  
don't have control over other people's repositories (i.e. they might  
be down when you need to checkout) it's probably a good idea to  
"checkpoint" specific releases and make sure they live in your repo  
somewhere.

-bob