provide a way for test collector to find custom test suites

[acapnotic]

We have some data-driven tests here, which we turn in to unittest-compatible tests by writing a bit of code to stuff them in to TestSuites. It'd be nice if there were a way to expose these custom TestSuites to trial's test collector.

Last time I chatted with people about this, there wasn't clearly an existing convention for that. It probably ends up looking something like "if a test module has a function with the magic name createTests(), trial runs it and collects the returned tests or suites."

(You can probably convince me to provide a patch. Let me know if there are behaviors that are particularly desirable or unacceptable.)

[spiv]

I recommend "test_suite" rather than "createTests", as that's an existing convention used by e.g. the Zope 3 test runner. I agree that Trial should support custom suites one way or another.

[jml]

I think this is done in source:branches/suite-1638. spiv, wanna revie?

[acapnotic]

I have a module with both standard TestCases and custom TestSuites, the r16711 will only call test_suite, not pick up the standard TestCases. well, that's okay, as long as I can use a test loader in my definition of test_suite...

but calling loadModule in r16711 would just result in calling test_suite again which would call loadModule until recursion limit reached.

[glyph]

Review tickets should be "highest" priority so they don't languish.

[radix]

I can see this being pretty useful, and the change is fine as far as changes go, but I'm really not sure about breaking the coding standard with "test_suite". Is this a Pyunit compatibility thing?

[spiv]

Zope 3 and bzr's test runners both look for "test_suite" functions in modules. I'm not sure if there's a deeper reason for that or not, but it does seem to be at least a bit of a precedent.

If we do something else in Trial, I'd still like there to be an easy way to make Trial look for "test_suite" functions, so I can use it with Zope 3 and bzr tests...

[jerub]

This isn't documented anywhere. I would prefer this wasn't merged until this is documented. Please add docstrings to the tests, and please add a docstring to loadModule.

I tried to find the docs for writing tests, but the only thing I found in svn was trunk/doc/core/howto/policy/test-standard.xhtml and that document isn't appropriate for documenting this change I don't feel. Perhaps a new ticket would be appropriate for the writing of a new document detailing the different ways it's possible to write unit tests.

I like how simple the change is. It's really quite neat and simple.

Be warned, merging will cause a conflict because other functions were added to the end of the file. you'll have to remove the <<<, === and >>> lines before commiting. Please make sure to run trial twisted.trial to make sure before committing.

[acapnotic]

merged forward to source:branches/suite-1638-2, docstrings added.

[radix]

Why is this assigned to me? It looks like Jerub did a review and forgot to un-mark the "review" tag and reassign it to the author.

[acapnotic]

(In [17821]) merge suite-1638-2. Trial discovers test_suite()

Author: jml, keturn Reviewer: jerub Fixes #1638

Adds the ability to trial to override test discovery for a module. If need to specially construct a test suite for your module (e.g. to create data-driven test cases), you can do so in a function named test_suite().

The name "test_suite" was chosen from its precedence in the Zope3 and bzr test runners.