Opened 11 years ago

Closed 10 years ago

#1638 enhancement closed fixed (fixed)

provide a way for test collector to find custom test suites

Reported by: acapnotic Owned by:
Priority: highest Milestone:
Component: trial Keywords:
Cc: spiv Branch:


We have some data-driven tests here, which we turn in to unittest-compatible tests by writing a bit of code to stuff them in to TestSuites. It'd be nice if there were a way to expose these custom TestSuites to trial's test collector.

Last time I chatted with people about this, there wasn't clearly an existing convention for that. It probably ends up looking something like "if a test module has a function with the magic name createTests(), trial runs it and collects the returned tests or suites."

(You can probably convince me to provide a patch. Let me know if there are behaviors that are particularly desirable or unacceptable.)

Change History (15)

comment:1 Changed 11 years ago by spiv

Cc: spiv added

I recommend "test_suite" rather than "createTests", as that's an existing convention used by e.g. the Zope 3 test runner. I agree that Trial should support custom suites one way or another.

comment:2 Changed 11 years ago by Jonathan Lange

Keywords: review added
Owner: changed from Jonathan Lange to spiv

I think this is done in source:branches/suite-1638. spiv, wanna revie?

comment:3 Changed 11 years ago by acapnotic

I have a module with both standard TestCases and custom TestSuites, the r16711 will only call test_suite, not pick up the standard TestCases. well, that's okay, as long as I can use a test loader in my definition of test_suite...

but calling loadModule in r16711 would just result in calling test_suite again which would call loadModule until recursion limit reached.

comment:4 Changed 11 years ago by Jonathan Lange

Keywords: review removed
Owner: changed from spiv to Jonathan Lange

comment:5 Changed 11 years ago by Jonathan Lange

Keywords: review added
Owner: changed from Jonathan Lange to spiv

comment:6 Changed 11 years ago by Glyph

Priority: normalhighest

Review tickets should be "highest" priority so they don't languish.

comment:7 Changed 11 years ago by Jonathan Lange

Owner: changed from spiv to radix

comment:8 Changed 11 years ago by radix

I can see this being pretty useful, and the change is fine as far as changes go, but I'm really not sure about breaking the coding standard with "test_suite". Is this a Pyunit compatibility thing?

comment:9 Changed 11 years ago by spiv

Zope 3 and bzr's test runners both look for "test_suite" functions in modules. I'm not sure if there's a deeper reason for that or not, but it does seem to be at least a bit of a precedent.

If we do something else in Trial, I'd still like there to be an easy way to make Trial look for "test_suite" functions, so I can use it with Zope 3 and bzr tests...

comment:10 Changed 11 years ago by Stephen Thorne

Keywords: review removed

This isn't documented anywhere. I would prefer this wasn't merged until this is documented. Please add docstrings to the tests, and please add a docstring to loadModule.

I tried to find the docs for writing tests, but the only thing I found in svn was trunk/doc/core/howto/policy/test-standard.xhtml and that document isn't appropriate for documenting this change I don't feel. Perhaps a new ticket would be appropriate for the writing of a new document detailing the different ways it's possible to write unit tests.

I like how simple the change is. It's really quite neat and simple.

Be warned, merging will cause a conflict because other functions were added to the end of the file. you'll have to remove the <<<, === and >>> lines before commiting. Please make sure to run trial twisted.trial to make sure before committing.

comment:11 Changed 10 years ago by acapnotic

Keywords: review added

merged forward to source:branches/suite-1638-2, docstrings added.

comment:12 Changed 10 years ago by radix

Owner: changed from radix to acapnotic

Why is this assigned to me? It looks like Jerub did a review and forgot to un-mark the "review" tag and reassign it to the author.

comment:13 Changed 10 years ago by radix

Keywords: review removed

comment:14 Changed 10 years ago by acapnotic

Resolution: fixed
Status: newclosed

(In [17821]) merge suite-1638-2. Trial discovers test_suite()

Author: jml, keturn Reviewer: jerub Fixes #1638

Adds the ability to trial to override test discovery for a module. If need to specially construct a test suite for your module (e.g. to create data-driven test cases), you can do so in a function named test_suite().

The name "test_suite" was chosen from its precedence in the Zope3 and bzr test runners.

comment:15 Changed 6 years ago by <automation>

Owner: acapnotic deleted
Note: See TracTickets for help on using tickets.