Opened 4 years ago

Closed 4 years ago

#9267 defect closed fixed (fixed)

SynchronousTestCase does complete test lifecycle after failure.

Reported by: mark williams Owned by: mark williams
Priority: normal Milestone:
Component: trial Keywords:
Cc: Allison Kaptur Branch:
Author:

Description

twisted.trial.unittest.SynchronousTestCase does not run tearDown or flush logged errors when a test method raises an exception:

from twisted.trial import unittest
from twisted.logger import Logger
from twisted.python.failure import Failure

class UnderTest(object):
    log = Logger()

    def method(self):
        try:
            raise Exception
        except:
            self.log.failure("what", failure=Failure())


class ExampleTest(unittest.SynchronousTestCase):

    def tearDown(self):
        self.assertFalse(True, "teardown Ran")

    def test_works(self):
        pass

    def test_broken(self):
        UnderTest().method()
        raise Exception("Failure")

This results in the following output:

module
  ExampleTest
    test_broken ...                                                     [ERROR]
    test_works ...                                                       [FAIL]
                                                     [ERROR]

===============================================================================
[FAIL]
Traceback (most recent call last):
  File "/path/to/module.py", line 18, in tearDown
    self.assertFalse(True, "teardown Ran")
  File "/home/mrw/.virtualenvs/path/to-f9530a6b996832d1/lib/python2.7/site-packages/twisted/trial/_synctest.py", line 384, in assertFalse
    super(_Assertions, self).assertFalse(condition, msg)
  File "/home/mrw/.pyenv/versions/2.7.13/lib/python2.7/unittest/case.py", line 416, in assertFalse
    raise self.failureException(msg)
twisted.trial.unittest.FailTest: teardown Ran

module.ExampleTest.test_works
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/path/to/module.py", line 25, in test_broken
    raise Exception("Failure")
exceptions.Exception: Failure

module.ExampleTest.test_broken
===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/path/to/module.py", line 10, in method
    raise Exception
exceptions.Exception: 

<not in test>
-------------------------------------------------------------------------------
Ran 2 tests in 0.126s

test_works runs the tearDown method, resulting in the assertion failure, while test_broken's lifecycle ends with the raised exception, so that tearDown does not run and the logged exception is collected by the runner.

The reason is that the tear down phase of a test's lifecycle does not happen in a finally clause. This is particularly bad with logged errors, because the observer is global mutable state. That means errors will hang around until something flushes those errors out.

Note that this also means an exception raised in tearDown will prevent flush logs from appearing with their test.

The tearDown method should be called and logged errors flushed even when the test method raises an exception.

See https://github.com/twisted/treq/issues/205 - thanks to Allison Kaptur for reporting this!

Change History (5)

comment:1 Changed 4 years ago by mark williams

Note that the documentation says tearDown should be called even when a test fails:

Note that a test class may also provide the counterpart of setUp , named tearDown , which will be called after each test (whether successful or not).

comment:2 Changed 4 years ago by mark williams

Keywords: review added

comment:3 Changed 4 years ago by Allison Kaptur

Cc: Allison Kaptur added

comment:4 Changed 4 years ago by Tristan Seligmann

Keywords: review removed
Owner: set to mark williams

comment:5 Changed 4 years ago by Mark Williams <mrw@…>

Resolution: fixed
Status: newclosed

In 33db6d7b:

Error: Processor CommitTicketReference failed
 does not appear to be a Git repository. See the log for more information.
Note: See TracTickets for help on using tickets.