[Twisted-Python] Subproject releases
James Y Knight
foom at fuhm.net
Mon Oct 17 22:04:06 MDT 2005
On Oct 12, 2005, at 10:29 PM, Glyph Lefkowitz wrote:
> Hmm. Trial interaction is something I haven't messed with yet; I
> haven't put trial directory below a __path__ entry.
>
> Arguably this is simply a bug in Trial that should be fixed, though.
>
The main problem here is that trial's "recursively find all test
modules for me" mode is based upon a recursive walk of the filesystem
below a certain path. Currently, "trial twisted" does the following:
1) import the module named ("twisted")
2) os.path.walk over all files under the directory of that module's
__file__ ("twisted/")
3) filter out (a) any directory which doesn't contain an __init__.*,
and (b) any files that don't match test_*.py.
4) backwards convert the path of the matched files to a module name
with some poor heuristics (reflect.filenameToModuleName).
5) try to import said module.
This completely breaks down in the case where packages use __path__.
The new hierarchy has the following:
Root/
Root/twisted/
Root/twisted/__init__.py (has __path__="core/twisted/","web2/twisted/")
core/
core/twisted/
core/twisted/__init__.py
core/twisted/test/
core/twisted/test/__init__.py
core/twisted/test/test_tcp.py
web2/
web2/twisted/
web2/twisted/web2/
web2/twisted/web2/__init__.py
web2/twisted/web2/test/
web2/twisted/web2/test/__init__.py
web2/twisted/web2/test/test_http.py
So, given that layout, currently, "trial twisted" would get
twisted.__file__ == "Root/twisted/__init__.py", take the dirname,
"Root/twisted/", and walk that hierarchy looking for tests (and there
aren't any there. Oops). Okay, let's specify the directory explicitly
-- I'll say "trial web2/twisted/web2/test/". Now, that doesn't work
right either, because it's impossible to backwards convert an
arbitrary filename to the python module name it's expecting to be
imported under. filenameToModuleName in this case would return
"web2.test.test_http", which is wrong.
So, my solution:
I changed the recursive process to actually import every package (not
every module!), so it can use the package's __path__ attribute to
find submodules. This works, mostly.
One issue is that it causes trial to emit an import error message
from trying to import twisted.internet.iocpreactor and
twisted.internet.serialport (because they aren't importable on my
system). This is necessary -- trial does not know they don't have a
__path__ attribute before importing them, so it has no way of telling
that they don't have test modules hidden inside. It would similarly
be a bad idea to simply ignore all import errors of packages, as then
real errors may be silenced. This could be simply solvable via a
special attribute like "no_tests_in_packages= ['iocpreactor',
'serialport']" to twisted/internet/__init__ so trial knows not to
bother trying to import those.
The other issue is that test files expecting to be imported as
modules _still_ won't work if specified by filename. But, I think
that's just a fact of life -- it is not possible in general to do the
correct back conversion. AFAICT we will just have to accept this
slight functionality loss and recommend that everybody specify module
names rather than file names (at least for running twisted's tests --
it's only broken when filenameToModuleName is wrong). Also, Trial
*does* run the file specified even if it can't figure out how to
import it, so if the test code doesn't actually depend on having its
real module name (e.g. it doesn't use things like relative imports or
__name__), it will also still work.
James
More information about the Twisted-Python
mailing list