Ticket #2739 (closed enhancement: fixed )

Opened 3 years ago

Last modified 2 years ago

explain, document, and test trial's compatibility requirements for other test tools

Reported by: glyph Assigned to: jml
Type: enhancement Priority: highest
Milestone: Component: trial
Keywords: Cc:
Branch: branches/trial-compatibility-doc-2739 Author: jml
Launchpad Bug:

Description

The issue of trial needing to be compatible with other test runners is periodically raised, for example, in the comments on #1870. I believe Zope's test runner is the most popular of these, but I'm not sure.

It seems as though there is little information available for those of us who might want to make changes to trial as to what changes are acceptable and what changes are not, with respect to this requirement.

Are there tests within trial which verify this compatibility? It looks like test_pyunitcompat is for running pyunit tests with trial, not trial tests with pyunit (or pyunit-like frameworks). If they're somewhere else, where are they? If not, can some be written? With some dependency detection, perhaps, so we don't cause an earthquake on all the buildbots at once?

Is there documentation within trial which explains what the requirements of "other test runners" are?

What is the list of supported, external test runners? Is this documented anywhere?

If all of these things exist, then this ticket could be resolved simply by making them easier to find. Placing a link on any linked twistedmatrix.com page tends to push stuff to the top of google results pretty fast. In searching around for a variety of phrases, this file in Zope3 SVN was the only thing I came across.

Attachments

Change History

  2007-07-20 14:05:16+00:00 changed by exarkun

  • priority changed from normal to highest

Thanks for filing this ticket.

Since this is often given as a reason for some change to be made or not to be made to trial, resolving this is probably the highest priority trial ticket right now. Any other potential bug fix or enhancement runs the risk of encountering a constraint which isn't documented or generally known.

  2007-07-25 00:00:57+00:00 changed by jml

Fair call.

Here is a rough draft:

  • Respect the interfaces defined in stdlib unittest.py
  • In particular, TestCase?.run(TestResult()) should 'work' (i.e. complete synchronously, populate the result object etc)
  • If trial foo.test_something.TestFoo.test_pass works, then TestFoo('test_pass').run(TestResult()) should also work, and vice versa.

  2007-11-16 19:41:05+00:00 changed by jml

(In [21827]) Define ITestCase to be the interface that test frameworks expect of a test.

  • Author: jml
  • Reviewers: therve, exarkun
  • Fixes #2898

ITestCase is re-purposed to define what you can expect from a test object. We use this interface to adapt pyunit tests to things usable in Trial.

In the process we also clarify what's needed for compatibility with doctest and pyunit.

This branch also fixes #1950.

Refs #2739.

  2007-11-20 18:51:10+00:00 changed by jml

  • branch set to branches/trial-compatibility-doc-2739
  • author set to jml

(In [21903]) Branching to 'trial-compatibility-doc-2739'

  2007-11-20 19:12:11+00:00 changed by jml

  • keywords set to review
  • owner deleted

  2007-11-20 22:07:37+00:00 changed by exarkun

  • keywords deleted
  • owner set to jml

The URLs could use U{} to make them links in the output. No need for re-review.

  2007-11-21 16:25:33+00:00 changed by jml

  • status changed from new to closed
  • resolution set to fixed

(In [21934]) Document Trial's compatibility requirements.

  • Author: jml
  • Reviewer: exarkun
  • Fixes #2739

Add information to the trial package docstring that starts to define the compatibility requirements for trial, giving examples.

Note: See TracTickets for help on using tickets.