trial's error reporting (especially --tbformat=emacs) does not like unicode exceptions

[marienz]

If I run "trial --tbformat=emacs test_wtf.py" on the following file:

from twisted.trial import unittest

class Borkage(unittest.TestCase):
    def test_borkage(self):
        raise ValueError(u'\xff')

I get:

test_wtf
  Borkage
    test_borkage ...                                                    [ERROR]

===============================================================================
[ERROR]: test_wtf.Borkage.test_borkage

Traceback (most recent call last):
  File "bin/trial", line 22, in <module>
    run()
  File "/home/marienz/src/twisted/twisted/scripts/trial.py", line 363, in run
    test_result = trialRunner.run(suite)
  File "/home/marienz/src/twisted/twisted/trial/runner.py", line 835, in run
    return self._runWithoutDecoration(test)
  File "/home/marienz/src/twisted/twisted/trial/runner.py", line 883, in _runWithoutDecoration
    result.done()
  File "/home/marienz/src/twisted/twisted/trial/reporter.py", line 636, in done
    self._printErrors()
  File "/home/marienz/src/twisted/twisted/trial/reporter.py", line 580, in _printErrors
    self._formatFailureTraceback)
  File "/home/marienz/src/twisted/twisted/trial/reporter.py", line 545, in _printResults
    self._write(formatter(*(content[1:])))
  File "/home/marienz/src/twisted/twisted/trial/reporter.py", line 527, in _formatFailureTraceback
    result = fail.getTraceback(detail=self.tbformat, elideFrameworkCode=True)
  File "/home/marienz/src/twisted/twisted/python/failure.py", line 472, in getTraceback
    self.printTraceback(file=io, elideFrameworkCode=elideFrameworkCode, detail=detail)
  File "/home/marienz/src/twisted/twisted/python/failure.py", line 492, in printTraceback
    w("%s: %s: %s\n" % (hasFrames, self.type, self.value))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xff' in position 0: ordinal not in range(128)

This is annoying because the line raising that exception is not actually in the reported traceback. Failures after this one also don't get reported anymore.

Without --tbformat=emacs the result is much better:

test_wtf
  Borkage
    test_borkage ...                                                    [ERROR]

===============================================================================
[ERROR]: test_wtf.Borkage.test_borkage

Traceback (most recent call last):
  File "/tmp//test_wtf.py", line 9, in test_borkage
    raise ValueError(u'\xff')
exceptions.ValueError: <ValueError instance at 0x97d294c with str error:
 Traceback (most recent call last):
  File "/home/marienz/src/twisted/twisted/python/reflect.py", line 546, in _safeFormat
    return formatter(o)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xff' in position 0: ordinal not in range(128)
>
-------------------------------------------------------------------------------
Ran 1 tests in 0.014s

FAILED (errors=1)

The problem is that if actual application code raises a weird exception like this one and you use --tbformat=emacs it is hard to figure out where the exception came from. Making trial more robust in reporting these would be nice.

[exarkun]

(In [29668]) Branching to 'failure-unicode-4501'

[exarkun]

This was caused by a cause of Failure.getTraceback which could raise an exception. Fixed in the branch, please review.

 Build results

[thijs]

Could you also update the docstring of the method you're updating (printTraceback), put it on 3 lines and add epytext for it's arguments? That way we can get a better understanding of the "brief" reference you make in the NEWS file. Personally I would also format all of the methods pep8-style but I know you're not a fan of doing that for a whole module :) Cheers.

[exarkun]

(In [29676]) Update printTraceback's docstring

refs #4501

[thijs]

Result of the test looks good:

$ bin/trial --tbformat=emacs test_wtf.py
test_wtf
  Borkage
    test_borkage ...                                                    [ERROR]

===============================================================================
[ERROR]: test_wtf.Borkage.test_borkage

Traceback: <type 'exceptions.ValueError'>: <ValueError instance at 0x103367d40 with str error:
 Traceback (most recent call last):
  File "/Volumes/Users/thijstriemstra/Sites/projects/opensource/Twisted/branches/failure-unicode-4501/twisted/python/reflect.py", line 546, in _safeFormat
    return formatter(o)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xff' in position 0: ordinal not in range(128)
>
/Volumes/Users/thijstriemstra/Sites/projects/opensource/Twisted/branches/failure-unicode-4501/test_wtf.py:5:test_borkage
-------------------------------------------------------------------------------
Ran 1 tests in 0.008s

FAILED (errors=1)

Except for the '>' character, it's a little confusing, maybe it should be moved behind the '0x103367d40' where it kinda belongs? Fine to merge though.

[exarkun]

(In [29694]) Mix object into BrokenExceptionType so that it works on Python 2.4

refs #4501

[exarkun]

Except for the '>' character, it's a little confusing, maybe it should be moved behind the '0x103367d40' where it kinda belongs?

It's tricky to move. The code pretty much just puts a traceback inside a '<%s>'. This is a general issue with safe_str and safe_repr though, so we can deal with it separately.

[exarkun]

(In [29695]) Merge failure-unicode-4501

Author: exarkun Reviewer: thijs Fixes: #4501

Fix the error handling in Failure.printTraceback in the "brief" detail mode so that exceptions which raise an exception when being converted to a string do not cause printTraceback itself to raise an exception.

[exarkun]

(In [29704]) Merge failure-unicode-4501

Author: exarkun Reviewer: thijs Fixes: #4501

Fix the error handling in Failure.printTraceback in the "brief" detail mode so that exceptions which raise an exception when being converted to a string do not cause printTraceback itself to raise an exception.

Re-merge includes the last revision from the branch which was accidentally omitted in the first merge.