[Twisted-web] Obvious stuff... just not to me, re: Failure to Install...

Jean-Paul Calderone exarkun at divmod.com
Sun Jun 17 19:54:44 EDT 2007

On Sun, 17 Jun 2007 19:25:29 -0400, Dav Clark <dav at securme.net> wrote:
>I'm slowly beginning to wrap my head around Nevow.  For one, while  it's not 
>explicit anywhere, it seems really clear that the intention  is simply to 
>add the Nevow root directory to your python path.

The Nevow install process is more or less the same as the install process
for any other Python package.  Yep, the package directories need to be in
sys.path somehow, either by having their parents added to PYTHONPATH or by
being copied to a directory which is in the default search path.

>In my  case, I'm using 
>the easy_install develop workaround:
>python -c "import setuptools; execfile('setup.py')" develop
>from the .../Nevow path.  This means I can svn update, and that  version is 
>working without any more reinstalls.

This may indeed work, although setuptools isn't really supported (meaning
there is no automated test coverage for setuptools working, and further none
of the primary Nevow developers use setuptools, so if it is working now, we
might break it unintentionally and unknowingly at some future time).

I use Combinator to manage my Python import path.  If you're tracking svn
trunk at HEAD, you might want to consider it too.  Or you can just edit your
shell's startup script to add the checkout directory to PYTHONPATH.

>In addition, I got thrown for a loop with the Athena tutorial,  because it 
>suggests that to understand the Nevow plugin search  system, you should just 
>look at the Twisted plugin search system.   This had me with a custom 
><emph>twisted</emph>/plugins directory, and  using pdb to step through code 
>to see why it wasn't getting loaded by  Nevow, when clearly the Twisted 
>plugin system was populating this  with the normal 'dropin.cache' and 
>But now I understand that the nevow/plugins and the twisted/plugins  are 
>similarly handled but distinct repo's.

Oops, that sounds like some unfortunately misleading documentation.  Do you
think you could correct the wiki page in question to make this explicit?

>So, while I have a borderline unhealthy desire to learn twisted &  nevow, 
>and will work through this stuff, I recon a lot of people  might have a look 
>and say, "WTF!" and go to Django or something.  I'd  like to help fix the 
>documentation as I learn, but I don't want to  write the wrong thing either. 
>So, here are some changes I would make  if OK to the Nevow and Athena docs 
>on the Wiki:
>1) This is the only obviously wrong thing - on the Athena page, it  says 
>that you should install modules into a plugin directory under  Nevow instead 
>of nevow.  This is confusing and had me creating a  Nevow/plugins directory 
>in my private PYTHONPATH (which makes sense  at first, as there is a Nevow 
>directory as well as a nevow directory  if you checkout from svn).

Aha.  Yes, please do fix this. :)

>2) Modify the Nevow install instructions to explain that neither ./ setup.py 
>install or easy_install are fully supported.  Suggest  easy_install develop 
>or similar as preferred alternative, outright  copying or adding the Nevow 
>direcotry to your PYTHONPATH as a  secondary (as well as setting up scripts 
>somewhere in your PATH).  I  would just fix the install to work properly, 
>but I'm not sure what  would be "right" more on that below.

setup.py is basically fully supported.  If you find bugs in it, we will do
our best to fix them.  easy_install isn't really supported at all; please
don't suggest it or anything related to setuptools.

>3) Regarding, e.g. javascript plugins, it seems that the convention  is to 
>install both user and 3rd party packages into the nevow  package.  This is 
>very different than what most people are used to,  but quite similar to the 
>way e.g. an in-place Zope install behaves  (particularly if you are doing an 
>in-place install as I describe  above).  I don't like doing this - as it 
>moves me away from a clean  separation of "my" code from "your" code.
>3a) So, for example, after getting the thing working with the stock 
>nevow/plugins directory, I standardized on having a local nevow/ plugins 
>directory as a sibling of my tac file.  I would suggest this  be the 
>recommended (or at least an alternative) method described in  the wiki - 
>this keeps all example files bundled under the same root  dir.  This also 
>allows the possibility of bundling a tar or zip of an  athena package, you 
>un-archive it, and the example just works with  twistd -ny example.tac from 
>where it sits.  (well, you need to alter  the /path/to/your/file to 
>sibpath(__file__, 'mymodule.js')).

I'm not totally clear on what you're suggesting here.  As described on the
wiki, the plugin system does let you keep all of your source files beneath
one directory.  Maybe I'm not understanding your concern, or maybe you're
still missing something about how plugins work.  Could you describe this
issue further?

>3b)  I would go a step further and suggest that, as in zope, a way to 
>create "instances" be created.  This would just mean populating a  directory 
>with things like a plugins direcotry, a js dir for 
>nevow.athena.autoJSPackage to load and other stuff I haven't learned  about 
>yet.  A set of scripts localized for that instance would add  the 
>appropriate root to the python path before running.  Or, you  could just 
>require people to run twistd from the root dir of the  bundle.  This would 
>be remarkably similar to the "solution" for 3a.

Requiring extra programs in order to run the actually interesting program
doesn't interest me very much.  Beyond this, I'm not really sure how this
suggestion differs much from the current system, except that it explicitly
makes environment setup something that goes into a helper script, rather
than relying on each user to have an environment which is "correctly"
configured.  Perhaps the documentation should go into more detail about
what Nevow considers a "correct" environment?

>3c) I'm still a bit unclear on how python deals with identically  named 
>packages in different directories in the path.  So, for  example, I know 
>have two nevow/plugins directories on my path.   Everything seems to work 
>fine, and indeed, the athena.JSDependencies  seems to be able to find files 
>from both.  Even if I add an  __init__.py file to my own nevow/plugins 
>directory, the autoJSPackage  call from the default 
>nevow/plugins/__init__.py seems to fire  correctly.  But at some point, name 
>collisions are necessarily going  to be a problem.  I could see some _very_ 
>funny errors arising from  this, e.g. if the python interpreter simply uses 
>the first file it  finds, it could load a .py file from one directory, then 
>load an  __init__.py file from another directory which overrides the objects 
>loaded from that first .py file... pointers would be appreciated on  this.

The plugin system uses some advanced features of the Python package system.
Normally Python packages wouldn't let you have multiple packages or modules
with the same name (and even here it doesn't really - the nevow/plugins and
nevow directories you create to add plugins are just directories, rather than
packages, although nevow/plugins/ does _contribute_ to "the" nevow.plugins

There is necessarily the potential for a name conflict here, since multiple
authors are contributing to the same namespace.  This is the same as the
problem of top-level Python packages, since two authors might choose to name
their package the same thing.  However, unlike that case, it is trivially
resolved, since the names of things in nevow/plugins/ are essentially
irrelevant.  At packaging or installation time, the name can be changed to
resolve a conflict without breaking the plugins (this might be a problem
for unit tests, though, if they try to import from the plugins package in
order to test functionality it provides - they will be broken by the name
change, probably).

>3d) Speaking of strange .js includes, some system (correctly?)  associates 
>the nevow/empty.js file with most Nevow classes.  But for  root classes, you 
>also get these __init__.js associations, which are  non-existent *.js files. 
>Is this a bug?

Hmm, could be.  Can you describe this further?

>4) Perhaps have a page describing Divmod wiki etiquette... answering  the 
>question of this e-mail, which is "How liberal should I be in  editing the 
>5) Why do all examples have twistd -noy, when -y implies -o?

Historic reasons.  The people writing these wiki pages probably still have
finger memory from before -y implied -o. :)  Feel free to fix this.

>Anyway, as I've said, I'd like to leave some breadcrumbs for those  right 
>behind me before I forget what it's like to be such a noob!   Any feedback 
>or guidance on appropriate help is appreciated.  In  particular, how I 
>should proceed now and in future with Wiki /  documentation edits.

Thanks for digging into this and collecting your experiences in this email!


More information about the Twisted-web mailing list