From: <a href="mailto:exarkun@twistedmatrix.com">exarkun@twistedmatrix.com</a><br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Subject: Re: [Twisted-Python] Adding mock as a test suite dependency<br><br>
On 03:00 am, <a href="mailto:glyph@twistedmatrix.com">glyph@twistedmatrix.com</a> wrote:<br>>Also I don't particularly like the testing style associated with Mock.<br>
>I think it might discourage us yet further from writing verified fakes,<br>
>i.e. supported in-memory implementations of things like IReactorTCP,<br>
>that have somewhat intricate behavior that's tedious to emulate with<br>
>Mock.<br>
<br>
I'm also not a huge fan of the *unverified* mock style of testing. I<br>
don't think anything says that mocks *have* to be unverified, though it<br>
seems they're often used that way.<br>
<br>
The mock library that got added to the stdlib has the notion of<br>
constructing a mock using another object as a template. I haven't used<br>
this feature, but it seems like the intent is to at least take a step<br>
towards verification. It'd be nice if someone who knows more about the<br>
features of this library could give some examples.<br>
<br>
In case anyone isn't clear, the problem with unverified fakes is that<br>
they either start out incompatible with the objects they're fakes of, or<br>
else they become incompatible with them over time. Once they're<br>
incompatible, the tests that use them become significantly less useful,<br>
since they demonstrate little or nothing about what will happen when you<br>
try to use the code for real.<br>
<br>
Verified fakes solve this problem by adding assertions that objects and<br>
their fakes have the necessary overlap in either interface or<br>
functionality in order for the tests using them to be valid.<br>
<br>
Beyond that, considering the particular example presented, I wouldn't<br>
actually use mocks to test this. The real object, the debugger, should<br>
be perfectly usable in unit tests. It doesn't allocate or depend on<br>
expensive resources, it doesn't do network I/O, etc. Mocks are perhaps<br>
an attractive nuisance that distract from coming up with a better test.<br>
<br></blockquote><div><br></div><div>Well it seems I sidestepped behavior vs. state but fell into isolationist vs integration :).</div><div><br></div><div>The reason I think even this was a reasonable example is because in the code that this</div>
<div>will test, there is no dependence on an actual debugger whatsoever. What this test should</div><div>be testing is that an object whose interface is irrelevant for the purpose of the test was</div><div>handed off to another method in the case that that is expected.</div>
<div><br></div><div>In short, the isolationist view as I understand and have come to appreciate says that</div><div>mocks (in a broader sense here since these aren't true mocks I guess) aren't just for</div><div>cases where the real object is expensive or annoying to create – they also remove</div>
<div>irrelevant details from the body of the test.</div><div><br></div><div>That being said though, I'm still looking (read: I have not yet looked but will do so when I</div><div>get home) for actual examples in the test suite I can point to and say that real, actual mocks</div>
<div>would have helped if that's what you'd be looking for.</div><div> </div><div>To go back to your first point about verification, mock has a bunch of things there of which</div><div>if I'm truthful I only use some of them with any regularity. The thing that sounds like what</div>
<div>you're referring to is likely the `spec` argument, which will do something like:</div><div><br></div><div><div>>>> import mock</div><div>>>> from twisted.trial.itrial import ITestCase</div><div>
>>> testCase = mock.Mock(spec=ITestCase.names())</div><div>>>> testCase.run</div><div><Mock name='mock.run' id='4319309392'></div><div>>>> testCase.jump</div><div>Traceback (most recent call last):</div>
<div> File "<input>", line 1, in <module></div><div> File "/usr/local/Cellar/python/2.7.3/lib/python2.7/site-packages/mock.py", line 647, in __getattr__</div><div> raise AttributeError("Mock object has no attribute %r" % name)</div>
<div>AttributeError: Mock object has no attribute 'jump'</div></div><div><br></div><div><br></div><div>As someone mentioned you also can hand it an implementor of ITestCase too but it will then pull</div><div>off all attributes that that object has.</div>
<div><br></div><div>There's plenty more granularity, but like anything else the design of the test requires common sense.</div><div>If the test is meant to test how a specific method on the testCase mock is being used then assertions</div>
<div>on how it was called generally seem most natural.</div><div><br></div><div>As an alternate approach, I've learned (from a few places I think) that in cases where I want to verify</div><div>that the places I've mocked are being used in a way that isn't going to differ from their actual use in the</div>
<div>code, that a much wider scoped test that does integrate but doesn't need to be a unit test can be helpful,</div><div>such that there are a whole bunch of isolated tests for each piece of logic and then one integrated test</div>
<div>that actually does push the real object through such that anyone who does change the way that the two</div><div>objects interacts still will have something telling them that the tests need updating as they would if all of</div>
<div>the tests integrated.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Jean-Paul<br>
>Personally I'm -0. Don't let that stop you from cooking up a patch<br>
>that would include it though, I might be in the minority here.<br>
><br>
>-glyph<br></blockquote></div>