[Twisted-web] Obvious stuff... just not to me,
re: Failure to Install...
exarkun at divmod.com
Sun Jun 17 19:54:44 EDT 2007
On Sun, 17 Jun 2007 19:25:29 -0400, Dav Clark <dav at securme.net> wrote:
>I'm slowly beginning to wrap my head around Nevow. For one, while it's not
>explicit anywhere, it seems really clear that the intention is simply to
>add the Nevow root directory to your python path.
The Nevow install process is more or less the same as the install process
for any other Python package. Yep, the package directories need to be in
sys.path somehow, either by having their parents added to PYTHONPATH or by
being copied to a directory which is in the default search path.
>In my case, I'm using
>the easy_install develop workaround:
>python -c "import setuptools; execfile('setup.py')" develop
>from the .../Nevow path. This means I can svn update, and that version is
>working without any more reinstalls.
This may indeed work, although setuptools isn't really supported (meaning
there is no automated test coverage for setuptools working, and further none
of the primary Nevow developers use setuptools, so if it is working now, we
might break it unintentionally and unknowingly at some future time).
I use Combinator to manage my Python import path. If you're tracking svn
trunk at HEAD, you might want to consider it too. Or you can just edit your
shell's startup script to add the checkout directory to PYTHONPATH.
>In addition, I got thrown for a loop with the Athena tutorial, because it
>suggests that to understand the Nevow plugin search system, you should just
>look at the Twisted plugin search system. This had me with a custom
><emph>twisted</emph>/plugins directory, and using pdb to step through code
>to see why it wasn't getting loaded by Nevow, when clearly the Twisted
>plugin system was populating this with the normal 'dropin.cache' and
>But now I understand that the nevow/plugins and the twisted/plugins are
>similarly handled but distinct repo's.
Oops, that sounds like some unfortunately misleading documentation. Do you
think you could correct the wiki page in question to make this explicit?
>So, while I have a borderline unhealthy desire to learn twisted & nevow,
>and will work through this stuff, I recon a lot of people might have a look
>and say, "WTF!" and go to Django or something. I'd like to help fix the
>documentation as I learn, but I don't want to write the wrong thing either.
>So, here are some changes I would make if OK to the Nevow and Athena docs
>on the Wiki:
>1) This is the only obviously wrong thing - on the Athena page, it says
>that you should install modules into a plugin directory under Nevow instead
>of nevow. This is confusing and had me creating a Nevow/plugins directory
>in my private PYTHONPATH (which makes sense at first, as there is a Nevow
>directory as well as a nevow directory if you checkout from svn).
Aha. Yes, please do fix this. :)
>2) Modify the Nevow install instructions to explain that neither ./ setup.py
>install or easy_install are fully supported. Suggest easy_install develop
>or similar as preferred alternative, outright copying or adding the Nevow
>direcotry to your PYTHONPATH as a secondary (as well as setting up scripts
>somewhere in your PATH). I would just fix the install to work properly,
>but I'm not sure what would be "right" more on that below.
setup.py is basically fully supported. If you find bugs in it, we will do
our best to fix them. easy_install isn't really supported at all; please
don't suggest it or anything related to setuptools.
>install both user and 3rd party packages into the nevow package. This is
>very different than what most people are used to, but quite similar to the
>way e.g. an in-place Zope install behaves (particularly if you are doing an
>in-place install as I describe above). I don't like doing this - as it
>moves me away from a clean separation of "my" code from "your" code.
>3a) So, for example, after getting the thing working with the stock
>nevow/plugins directory, I standardized on having a local nevow/ plugins
>directory as a sibling of my tac file. I would suggest this be the
>recommended (or at least an alternative) method described in the wiki -
>this keeps all example files bundled under the same root dir. This also
>allows the possibility of bundling a tar or zip of an athena package, you
>un-archive it, and the example just works with twistd -ny example.tac from
>where it sits. (well, you need to alter the /path/to/your/file to
I'm not totally clear on what you're suggesting here. As described on the
wiki, the plugin system does let you keep all of your source files beneath
one directory. Maybe I'm not understanding your concern, or maybe you're
still missing something about how plugins work. Could you describe this
>3b) I would go a step further and suggest that, as in zope, a way to
>create "instances" be created. This would just mean populating a directory
>with things like a plugins direcotry, a js dir for
>nevow.athena.autoJSPackage to load and other stuff I haven't learned about
>yet. A set of scripts localized for that instance would add the
>appropriate root to the python path before running. Or, you could just
>require people to run twistd from the root dir of the bundle. This would
>be remarkably similar to the "solution" for 3a.
Requiring extra programs in order to run the actually interesting program
doesn't interest me very much. Beyond this, I'm not really sure how this
suggestion differs much from the current system, except that it explicitly
makes environment setup something that goes into a helper script, rather
than relying on each user to have an environment which is "correctly"
configured. Perhaps the documentation should go into more detail about
what Nevow considers a "correct" environment?
>3c) I'm still a bit unclear on how python deals with identically named
>packages in different directories in the path. So, for example, I know
>have two nevow/plugins directories on my path. Everything seems to work
>fine, and indeed, the athena.JSDependencies seems to be able to find files
>from both. Even if I add an __init__.py file to my own nevow/plugins
>directory, the autoJSPackage call from the default
>nevow/plugins/__init__.py seems to fire correctly. But at some point, name
>collisions are necessarily going to be a problem. I could see some _very_
>funny errors arising from this, e.g. if the python interpreter simply uses
>the first file it finds, it could load a .py file from one directory, then
>load an __init__.py file from another directory which overrides the objects
>loaded from that first .py file... pointers would be appreciated on this.
The plugin system uses some advanced features of the Python package system.
Normally Python packages wouldn't let you have multiple packages or modules
with the same name (and even here it doesn't really - the nevow/plugins and
nevow directories you create to add plugins are just directories, rather than
packages, although nevow/plugins/ does _contribute_ to "the" nevow.plugins
There is necessarily the potential for a name conflict here, since multiple
authors are contributing to the same namespace. This is the same as the
problem of top-level Python packages, since two authors might choose to name
their package the same thing. However, unlike that case, it is trivially
resolved, since the names of things in nevow/plugins/ are essentially
irrelevant. At packaging or installation time, the name can be changed to
resolve a conflict without breaking the plugins (this might be a problem
for unit tests, though, if they try to import from the plugins package in
order to test functionality it provides - they will be broken by the name
>3d) Speaking of strange .js includes, some system (correctly?) associates
>the nevow/empty.js file with most Nevow classes. But for root classes, you
>also get these __init__.js associations, which are non-existent *.js files.
>Is this a bug?
Hmm, could be. Can you describe this further?
>4) Perhaps have a page describing Divmod wiki etiquette... answering the
>question of this e-mail, which is "How liberal should I be in editing the
>5) Why do all examples have twistd -noy, when -y implies -o?
Historic reasons. The people writing these wiki pages probably still have
finger memory from before -y implied -o. :) Feel free to fix this.
>Anyway, as I've said, I'd like to leave some breadcrumbs for those right
>behind me before I forget what it's like to be such a noob! Any feedback
>or guidance on appropriate help is appreciated. In particular, how I
>should proceed now and in future with Wiki / documentation edits.
Thanks for digging into this and collecting your experiences in this email!
More information about the Twisted-web