Opened 8 years ago

Closed 8 years ago

#1638 enhancement closed fixed (fixed)

provide a way for test collector to find custom test suites

Reported by: acapnotic Owned by:
Priority: highest Milestone:
Component: trial Keywords:
Cc: spiv Branch:
Author: Launchpad Bug:

Description

We have some data-driven tests here, which we turn in to unittest-compatible tests by writing a bit of code to stuff them in to TestSuites. It'd be nice if there were a way to expose these custom TestSuites to trial's test collector.

Last time I chatted with people about this, there wasn't clearly an existing convention for that. It probably ends up looking something like "if a test module has a function with the magic name createTests(), trial runs it and collects the returned tests or suites."

(You can probably convince me to provide a patch. Let me know if there are behaviors that are particularly desirable or unacceptable.)

Change History (15)

comment:1 Changed 8 years ago by spiv

  • Cc spiv added

I recommend "test_suite" rather than "createTests", as that's an existing convention used by e.g. the Zope 3 test runner. I agree that Trial should support custom suites one way or another.

comment:2 Changed 8 years ago by jml

  • Keywords review added
  • Owner changed from jml to spiv

I think this is done in source:branches/suite-1638.
spiv, wanna revie?

comment:3 Changed 8 years ago by acapnotic

I have a module with both standard TestCases and custom TestSuites,
the r16711 will only call test_suite, not pick up the standard TestCases.
well, that's okay, as long as I can use a test loader in my definition of test_suite...

but calling loadModule in r16711 would just result in calling test_suite again which would call loadModule until recursion limit reached.

comment:4 Changed 8 years ago by jml

  • Keywords review removed
  • Owner changed from spiv to jml

comment:5 Changed 8 years ago by jml

  • Keywords review added
  • Owner changed from jml to spiv

comment:6 Changed 8 years ago by glyph

  • Priority changed from normal to highest

Review tickets should be "highest" priority so they don't languish.

comment:7 Changed 8 years ago by jml

  • Owner changed from spiv to radix

comment:8 Changed 8 years ago by radix

I can see this being pretty useful, and the change is fine as far as changes go, but I'm really not sure about breaking the coding standard with "test_suite". Is this a Pyunit compatibility thing?

comment:9 Changed 8 years ago by spiv

Zope 3 and bzr's test runners both look for "test_suite" functions in modules. I'm not sure if there's a deeper reason for that or not, but it does seem to be at least a bit of a precedent.

If we do something else in Trial, I'd still like there to be an easy way to make Trial look for "test_suite" functions, so I can use it with Zope 3 and bzr tests...

comment:10 Changed 8 years ago by jerub

  • Keywords review removed

This isn't documented anywhere. I would prefer this wasn't merged until this is documented. Please add docstrings to the tests, and please add a docstring to loadModule.

I tried to find the docs for writing tests, but the only thing I found in svn was trunk/doc/core/howto/policy/test-standard.xhtml and that document isn't appropriate for documenting this change I don't feel. Perhaps a new ticket would be appropriate for the writing of a new document detailing the different ways it's possible to write unit tests.

I like how simple the change is. It's really quite neat and simple.

Be warned, merging will cause a conflict because other functions were added to the end of the file. you'll have to remove the <<<, === and >>> lines before commiting. Please make sure to run trial twisted.trial to make sure before committing.

comment:11 Changed 8 years ago by acapnotic

  • Keywords review added

merged forward to source:branches/suite-1638-2,
docstrings added.

comment:12 Changed 8 years ago by radix

  • Owner changed from radix to acapnotic

Why is this assigned to me? It looks like Jerub did a review and forgot to un-mark the "review" tag and reassign it to the author.

comment:13 Changed 8 years ago by radix

  • Keywords review removed

comment:14 Changed 8 years ago by acapnotic

  • Resolution set to fixed
  • Status changed from new to closed

(In [17821]) merge suite-1638-2. Trial discovers test_suite()

Author: jml, keturn
Reviewer: jerub
Fixes #1638

Adds the ability to trial to override test discovery for a module. If
need to specially construct a test suite for your module (e.g. to
create data-driven test cases), you can do so in a function named
test_suite().

The name "test_suite" was chosen from its precedence in the Zope3 and
bzr test runners.

comment:15 Changed 4 years ago by <automation>

  • Owner acapnotic deleted
Note: See TracTickets for help on using tickets.