[Twisted-Python] Trial & the mock library

[exarkun at twistedmatrix.com]

On 25 Jul, 02:25 pm, jamesbroadhead at gmail.com wrote:
>Hey all -
>
>I've recently started working with the 'mock' library in our trial 
>tests,
>and am looking for some best-practice advice.  I'm really just starting 
>to
>get used to the library, so it might well have functionality that I'm
>unaware of or am misusing.
>
>I very quickly ran into a problem where I mistakenly returned a Mock() 
>in
>the place of a deferred, causing the asserts in callbacks to not be 
>called,
>and for the test to spuriously pass.

To address this problem, I suggest you get into the habit of watching 
your unit tests fail in the expected way before you make the necessary 
implementation changes to make them pass.

This is only one of an unlimited number of ways your unit tests can be 
buggy.  It might be tempting to try to fix the test runner to prevent 
you from ever falling into this trap again - and who knows, it might 
even be a good idea.
However, if you run your tests and see them fail in the way you expected 
them to fail before you write the code that makes them pass, then you 
will be sure to avoid the many, many, many *other* pitfalls that have 
nothing to do with accidentally returning the wrong object.

This is just one of the attractions of test-driven development for me.

Jean-Paul
>A basic example:
>
>def test_foo():
>  d = Mock()
>  def check_result(res):
>    self.assertEqual(res.code, expected)  # never called
>  d.addCallback(check_result)
>  return d # Mock is truthy, test passes
>
>This occurred where I was mocking some internals of the class under 
>test;
>something like the below
>
>A slightly more believable example:
>== myclass.py ==
>def some_function(...):
>  d = self.authenticate()
>  d.addCallback(foo) # foo never called
>  d.addErrback(bar) # bar never called
>  return d
>
>== test_myclass.py ==
>def setUp(self):
>  self.resource.authenticate = Mock(return_value=Mock())
>
>def test_foo():
>  d = self.resource.some_function
>  def check_result(res):  # never called
>    self.assertEqual(res.code, expected)
>  d.addCallback(check_result)
>  return d # test passes
>
>Currently, I'm experimenting with wrapping Mock instantiations by 
>defining
>common deferred methods on them in advance; this approach would 
>eventually
>lead to extending Mock itself with this functionality.
>
>def nonDeferredMock():
>    m = Mock()
>    def notimpl(*args, **kwargs):
>        raise NotImplementedError('You treated a Mock like a Deferred!')
>    m.addCallback = notimpl
>    m.addErrback = notimpl
>    m.addBoth = notimpl
>    m.addCallbacks = notimpl
>    return m
>
>Another approach might be extending TestCase to check that return 
>values
>are always not Mock objects.
>
>Does anyone on the list have experience with this? Obviously, this only
>happens when mistakes are made when writing tests, but I'd rather have
>confidence that when my tests pass, that they've passed for the right
>reasons.
>
>Another antipattern that I've come across has been:
>
>resource.mymethod = Mock(return_value=defer.succeed(None))
>
>which works fine for tests in which mymethod() is called once, but 
>always
>returns the same deferred object if multiple calls are made. What would 
>be
>a better approach?
>
>Cheers-
>
>James