Opened 12 years ago

Closed 7 years ago

Last modified 7 years ago

#4526 enhancement closed fixed (fixed)

Document how trial finds tests to run

Reported by: Screwtape Owned by: Adi Roiban
Priority: normal Milestone:
Component: trial Keywords: documentation easy
Cc: Thijs Triemstra, E. Dunham Branch: branches/trial-discovery-documentation-4526-2
branch-diff, diff-cov, branch-cov, buildbot
Author: adiroiban, rwall

Description

I've been on a bit of a trial documentation kick recently, and noticed that neither the man page nor the HTML docs discuss the algorithm trial uses when looking for test files.

For example, if I run:

trial my.python.package

then which of the following tests will be found?

  • TestCases defined in my/python/package/__init__.py
  • TestCases defined in my/python/package/test_foo.py
  • TestCases defined in my/python/package/Test_foo.py
  • TestCases defined in my/python/package/testfoo.py
  • TestCases defined in my/python/package/Testfoo.py
  • TestCases defined in my/python/package/foo_test.py

For freshly-written tests designed to work with Trial, I personally always use the test_foo.py variant, but sometimes when I want to use Trial's test-sniffing features with third-party libraries, or to test legacy code I've inherited, it would be nice to have a place where I could find out definitively what the rules are.

(yes, I suppose I could just run Trial and see, but sometimes it's easier to try nosetests or py.test or run things manually than decipher exactly what failure-mode caused a particular error message)

Attachments (2)

0001-Update-testing-with-how-Trial-finds-tests.patch (979 bytes) - added by E. Dunham 8 years ago.
document-how-trial-finds-tests.patch (906 bytes) - added by Adam Dangoor 8 years ago.
edunham's work with a NEWS file

Download all attachments as: .zip

Change History (21)

comment:1 Changed 12 years ago by Jean-Paul Calderone

Component: coretrial
Owner: changed from Glyph to Jonathan Lange

comment:2 Changed 11 years ago by <automation>

Owner: Jonathan Lange deleted

comment:3 Changed 11 years ago by Thijs Triemstra

Cc: Thijs Triemstra added

comment:4 Changed 9 years ago by Julian Berman

Keywords: easy added

comment:5 Changed 9 years ago by arbiter

Owner: set to arbiter
Status: newassigned

comment:6 Changed 9 years ago by arbiter

Owner: arbiter deleted
Status: assignednew

comment:7 Changed 8 years ago by E. Dunham

Cc: E. Dunham added
Owner: set to E. Dunham
Status: newassigned

The answer to this actually is in the docs, just hidden pretty well. Whether a test is run depends not on what file it's in, but on how your module is configured and what its class inherits from.

I guess that references to that documentation should be added to at least the documentation and possibly the man page as well.

comment:8 Changed 8 years ago by E. Dunham

The man page links the howto page from "see also", so no changes to the man page are required. Attached above is a patch which adds a link to the module loader docs from the testing howto.

comment:9 Changed 8 years ago by E. Dunham

Keywords: review added
Owner: E. Dunham deleted
Status: assignednew

comment:10 Changed 8 years ago by Adi Roiban

Keywords: review removed
Owner: set to E. Dunham

Thanks for the work on this.

The patch needs a news file. http://twistedmatrix.com/trac/wiki/ReviewProcess#Newsfiles

The preferred format is diff without prefix http://twistedmatrix.com/trac/wiki/GitMirror#CreatingaGitclone

Other than that, it looks good.

Thanks!

Changed 8 years ago by Adam Dangoor

edunham's work with a NEWS file

comment:11 Changed 8 years ago by Adam Dangoor

Keywords: review added
Owner: E. Dunham deleted

Hopefully I am not stepping on anyone's toes picking up the ticket edunham has worked on. I have addressed adiroiban's comments by adding a patch with a NEWS file and I used the patch-making instructions from https://twistedmatrix.com/trac/wiki/BasicGuideToContributingCode#SubmitaPatch.

comment:12 Changed 8 years ago by Richard Wall

Owner: set to Richard Wall
Status: newassigned

comment:13 Changed 8 years ago by Richard Wall

Author: rwall
Branch: branches/trial-discovery-documentation-4526

(In [43147]) Branching to trial-discovery-documentation-4526.

comment:14 Changed 8 years ago by Adi Roiban

All good! Please merge this! Thanks!

comment:15 Changed 7 years ago by Adi Roiban

Author: rwalladiroiban, rwall
Branch: branches/trial-discovery-documentation-4526branches/trial-discovery-documentation-4526-2

(In [43735]) Branching to trial-discovery-documentation-4526-2.

comment:16 Changed 7 years ago by Adi Roiban

Owner: changed from Richard Wall to Adi Roiban
Status: assignednew

I will sync with trunk and hope is ok... old branch was failing on buildbot

comment:17 Changed 7 years ago by Adi Roiban

Resolution: fixed
Status: newclosed

(In [43737]) Merge trial-discovery-documentation-4526-2: Update trial documentation.

Author: adamtheturtle Reviewers: adiroiban Fixes: #4526

Update trial documentation with link to how trial discover tests.

comment:18 Changed 7 years ago by Jean-Paul Calderone

Keywords: review removed

Don't forget to remove the review keyword when you review a ticket.

Also, the news fragment for this change seems to contain a mistake:

Twisted Trial's basics documentation now has a link to the documentation about how Trial finds tests.
Twisted Trial's basics documentation now has a link to the documentation about how Trial finds tests.

Separately, I'm not sure I'd consider this one line doc addition (which is just a reference to other existing documentation) non-misc material.

comment:19 Changed 7 years ago by Adi Roiban

hawkowl fixed this in #7335 as the duplicate news line was already merged. I am sorry for letting this pass the review and merged in trunk.

I will remove the review in the future. Thanks for the help!

Note: See TracTickets for help on using tickets.