Opened 16 years ago

Last modified 15 years ago

#1397 defect new

Trial reports SUCCESS!?! and ERROR for the same test

Reported by: jknight Owned by:
Priority: high Milestone:
Component: trial Keywords: waiting
Cc: Jonathan Lange, jknight, Jean-Paul Calderone, spiv Branch:


Change History (8)

comment:1 Changed 16 years ago by jknight

In the failing test I'm just about to commit: 
setting .todo doesn't actually work. 

Trial reports:
Running 1 tests.
    testCallLaterOrder2 ...                                        [SUCCESS!?!]

[ERROR]: twisted.test.test_internet.InterfaceTestCase.testCallLaterOrder2

Failure: twisted.trial.unittest.FailTest: [0, 2, 6, 12] != [0, 1, 2, 3, 4, 5, 6, 
7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
[SUCCESS!?!]: twisted.test.test_internet.InterfaceTestCase.testCallLaterOrder2

Reason: 'See bug 1396'

Ran 1 tests in 0.557s

comment:2 Changed 16 years ago by Jonathan Lange

testCallLaterOrder2 is managing to call log.err(). I guess this is because of
unhandled errors in callLaters.

So, this can be reduced to an earlier problem -- #1384

comment:3 Changed 16 years ago by Jonathan Lange

Cc: Jean-Paul Calderone spiv added

My last comment is mistaken.

The real problem is that log error calls are handled in cleanup, and todo has no authority there.

This creates two topics of interest:

# Should .todo cover the entire test, or just the test method? # log.err calls should be handled differently.

#1409 addresses the second point.

What about the first? (CC: opening up the discussion to other vocal trial users)

comment:4 Changed 16 years ago by Jonathan Lange

Keywords: waiting added

comment:5 Changed 15 years ago by jknight

Todo should cover the entire test.

comment:6 Changed 15 years ago by spiv

I think I agree with jknight that todo should cover the entire test.

In general, it's surprising any time multiple results are associated with a single test.

comment:7 Changed 15 years ago by Jonathan Lange

So, I'm happy now to make .todo cover the entire test.

However, as stated elsewhere (citation needed) I am committed to tests being able to have multiple results. Or rather, I am convinced that trial shouldn't hide errors, unless explicitly asked to (ala todo).

comment:8 Changed 11 years ago by <automation>

Owner: Jonathan Lange deleted
Note: See TracTickets for help on using tickets.