[Twisted-Python] Trial & the mock library

[James Broadhead]

Hey all -

I've recently started working with the 'mock' library in our trial tests,
and am looking for some best-practice advice.  I'm really just starting to
get used to the library, so it might well have functionality that I'm
unaware of or am misusing.

I very quickly ran into a problem where I mistakenly returned a Mock() in
the place of a deferred, causing the asserts in callbacks to not be called,
and for the test to spuriously pass.

A basic example:

def test_foo():
  d = Mock()
  def check_result(res):
    self.assertEqual(res.code, expected)  # never called
  d.addCallback(check_result)
  return d # Mock is truthy, test passes

This occurred where I was mocking some internals of the class under test;
something like the below

A slightly more believable example:
== myclass.py ==
def some_function(...):
  d = self.authenticate()
  d.addCallback(foo) # foo never called
  d.addErrback(bar) # bar never called
  return d

== test_myclass.py ==
def setUp(self):
  self.resource.authenticate = Mock(return_value=Mock())

def test_foo():
  d = self.resource.some_function
  def check_result(res):  # never called
    self.assertEqual(res.code, expected)
  d.addCallback(check_result)
  return d # test passes

Currently, I'm experimenting with wrapping Mock instantiations by defining
common deferred methods on them in advance; this approach would eventually
lead to extending Mock itself with this functionality.

def nonDeferredMock():
    m = Mock()
    def notimpl(*args, **kwargs):
        raise NotImplementedError('You treated a Mock like a Deferred!')
    m.addCallback = notimpl
    m.addErrback = notimpl
    m.addBoth = notimpl
    m.addCallbacks = notimpl
    return m

Another approach might be extending TestCase to check that return values
are always not Mock objects.

Does anyone on the list have experience with this? Obviously, this only
happens when mistakes are made when writing tests, but I'd rather have
confidence that when my tests pass, that they've passed for the right
reasons.

Another antipattern that I've come across has been:

resource.mymethod = Mock(return_value=defer.succeed(None))

which works fine for tests in which mymethod() is called once, but always
returns the same deferred object if multiple calls are made. What would be
a better approach?

Cheers-

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130725/6aa228d3/attachment.html>