twisted/plugins/__init__.py incorrectly considers other installations of Twisted

[radix]

If I have a system install of e.g. Twisted at /usr/lib/python2.4/site-packages/twisted, and a local install at ~/lib/python2.4/site-packages, and I make sure the latter is in my sys.path in such a way that "import twisted" picks up the copy in my home directory, plugin lookups will still honor plugins in the former location, to much sadness. It should probably not consider those plugins, since Python can't even import from that location.

[glyph]

This is one of the bugs fixed in the #1951 branch.

[radix]

In fact it is not fixed in #1951.

After further investigation, it has become clear that the problem does not lie in twisted.plugin, but rather twisted.plugin*s*, in its munging of path in init.py. It is adding entries to path from other installations of Twisted, when it should only be adding plugins directories of third-party applications: one (the only?) way to discern whether a plugins directory is a Twisted installation or a third-party application is to check to see if it has an init.py.

twisted/plugins/__init__.py

@@ -12 +12 @@
 """
 
 import os, sys
-__path__ = [os.path.abspath(os.path.join(x, 'twisted', 'plugins')) for x in sys.path]
+__path__.extend([os.path.abspath(os.path.join(x, 'twisted', 'plugins')) for x in sys.path if not os.path.exists(os.path.join(x, 'twisted', 'plugins', '__init__.py'))])
 
 __all__ = []                    # nothing to see here, move along, move along

[glyph]

I think that line has now gotten long enough to warrant a separate, tested function in twisted.python which multiple projects can use.

[exarkun]

/branches/plugin-collision-2339

[therve]

That looks good. pluginPackagePaths would need to have direct tests, to be sure of what is done in there.

AdjacentPackageTests?.createDummyPackage is missing a docstring.

[exarkun]

Fixed those things, both. Same branch.

[therve]

test_plugin.py is missing the Twisted copyright. Once done please merge.

[exarkun]

(In [20920]) Merge plugin-collision-2339

Author: exarkun Reviewer: therve Fixes #2339

Add a function, twisted.plugin.pluginPackagePaths, which returns a list of additional directories to search for plugin modules for a particular plugin package. Change twisted.plugins.init to use this function instead of looping over sys.path directly.

Unlike the loop previously in twisted.plugins.init, pluginPackagePaths will not include any directory which is a Python package. This prevents plugins from being discovered from packages which Python may not actually be able to import and limits plugin discovery to those installed on the version of the package which is in use by an application. It also allows a site installation to co-exist with another installation (for example, a personal user installation) without interfering with each other.

[itamarst]

Shouldn't the plugins howto also have been updated?

[exarkun]

I don't see anything about a plugin howto in the ticket description.

Want to file another ticket?

[itamarst]

I would argue that, as a rule, howtos should be updated on API changes, in that same ticket. However, in this case I opened a new ticket.