Opened 11 years ago

Closed 10 years ago

#2713 defect closed fixed (fixed)

trial -u can fail doctests on second pass

Reported by: mgeisler Owned by:
Priority: normal Milestone:
Component: trial Keywords:
Cc: Branch: branches/rerundoctest-2713
branch-diff, diff-cov, branch-cov, buildbot
Author: exarkun

Description

Given a file a.py containing a doctest like this:

def foo():
    """
    >>> foo()
    123
    """
    return 123

This doctest passes when used once with trial, but fails if trial is run with the --until-failure flag:

% trial -u trial_doctest_bug
Test Pass 1         
Running 1 tests.
doctest
  DocTestCase
    foo ...                                                            [OK]

---------------------------------------------------------------------------
Ran 1 tests in 0.030s

PASSED (successes=1)
Test Pass 2
Running 1 tests.
doctest
  DocTestCase
    foo ...                                                          [FAIL]

===========================================================================
[FAIL]: Doctest: trial_doctest_bug.a.foo

Traceback (most recent call last):
  File "unittest.py", line 260, in run
    testMethod()
  File "doctest.py", line 2157, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
exceptions.AssertionError: Failed doctest test for trial_doctest_bug.a.foo
  File "/tmp/trial_doctest_bug/a.py", line 2, in foo

----------------------------------------------------------------------
File "/tmp/trial_doctest_bug/a.py", line 5, in trial_doctest_bug.a.foo
Failed example:
    foo()
Exception raised:
    Traceback (most recent call last):
      File "doctest.py", line 1248, in __run
        compileflags, 1) in test.globs
      File "<doctest trial_doctest_bug.a.foo[0]>", line 1, in ?
        foo()
    NameError: name 'foo' is not defined

---------------------------------------------------------------------------
Ran 1 tests in 0.005s

FAILED (failures=1)

If I change the doctest so that it explicitly imports everything:

def foo():
    """
    >>> from trial_doctest_bug.a import foo
    >>> foo()
    123
    """
    return 123

then the doctest works when run repeatedly by trial.

Is this the indented behaviour?

Attachments (1)

DocTestCase-issue2604-workaround.patch (2.0 KB) - added by pjd 10 years ago.

Download all attachments as: .zip

Change History (15)

comment:1 Changed 11 years ago by mgeisler

I forgot to mention that I see this behavior with a fresh SVN checkout (revision 20548).

comment:2 Changed 10 years ago by pjd

Owner: changed from Jonathan Lange to pjd
Status: newassigned

This is a regression in doctest.DocTestCase in Python 2.4+; see issue2604.

Changed 10 years ago by pjd

comment:3 Changed 10 years ago by pjd

Keywords: review added
Owner: changed from pjd to Jean-Paul Calderone
Status: assignednew

The attached patch adds a workaround and test for the problem.

comment:4 Changed 10 years ago by Jean-Paul Calderone

author: exarkun
Branch: branches/rerundoctest-2713

(In [23672]) Branching to 'rerundoctest-2713'

comment:5 Changed 10 years ago by Jean-Paul Calderone

Resolution: fixed
Status: newclosed

(In [23683]) Merge rerundoctest-2713

Author: pjd Reviewer: exarkun Fixes: #2713

Work-around the behavior of DocTestSuite in Python 2.4 and newer which causes doctests to lose their globals on second and subsequent runs (eg during --until-failure runs) by saving and then restoring those globals.

comment:6 Changed 10 years ago by Jean-Paul Calderone

Resolution: fixed
Status: closedreopened

(In [23684]) Revert r23682 - Python 2.3 doctest regression

Reopens: #2713

Python 2.3 builder has a total failure inducing problem:

Traceback (most recent call last):
  File "./bin/trial", line 24, in ?
    run()
  File "/srv/bb-slave/Run/slave/full2.3/Twisted/twisted/scripts/trial.py", line 357, in run
    suite = _getSuite(config)
  File "/srv/bb-slave/Run/slave/full2.3/Twisted/twisted/scripts/trial.py", line 315, in _getSuite
    return loader.loadByNames(config['tests'], recurse)
  File "/srv/bb-slave/Run/slave/full2.3/Twisted/twisted/trial/runner.py", line 638, in loadByNames
    suites = [self.loadAnything(thing, recurse)
  File "/srv/bb-slave/Run/slave/full2.3/Twisted/twisted/trial/runner.py", line 596, in loadAnything
    return self.loadPackage(thing, recurse)
  File "/srv/bb-slave/Run/slave/full2.3/Twisted/twisted/trial/runner.py", line 551, in loadPackage
    thingToAdd = self.loadModule(module)
  File "/srv/bb-slave/Run/slave/full2.3/Twisted/twisted/trial/runner.py", line 475, in loadModule
    docSuite.addTest(self.loadDoctests(doctest))
  File "/srv/bb-slave/Run/slave/full2.3/Twisted/twisted/trial/runner.py", line 581, in loadDoctests
    return doctest.DocTestSuite(module, setUp=saveGlobs)
TypeError: DocTestSuite() got an unexpected keyword argument 'setUp'

comment:7 Changed 10 years ago by Jean-Paul Calderone

Keywords: review removed

Someone else should take a look at this now I suppose.

comment:8 Changed 10 years ago by Jean-Paul Calderone

Owner: Jean-Paul Calderone deleted
Status: reopenednew

comment:9 Changed 10 years ago by Jean-Paul Calderone

Keywords: review added

comment:10 Changed 10 years ago by therve

Owner: set to therve

comment:11 Changed 10 years ago by therve

Keywords: review removed
Owner: therve deleted

Looks fine, please merge.

comment:12 Changed 10 years ago by Jean-Paul Calderone

Owner: set to Jean-Paul Calderone
Status: newassigned

comment:13 Changed 10 years ago by Jean-Paul Calderone

Resolution: fixed
Status: assignedclosed

(In [23851]) Merge rerundoctest-2713

Author: pjd, exarkun Reviewer: therve Fixes: #2713

Work around a regression in doctest in Python 2.4 and newer which causes the globals for a test to be discarded after the first run. This prevents most doctests from ever succeeding more than once when run with trial's --until-failure option.

comment:14 Changed 8 years ago by <automation>

Owner: Jean-Paul Calderone deleted
Note: See TracTickets for help on using tickets.