[Twisted-Python] Testing Strategies (was Re: Streaming Requests)

[Jean-Paul Calderone]

On Mon, Jan 9, 2017 at 4:52 AM, Glyph Lefkowitz <glyph at twistedmatrix.com>
wrote:

> On Jan 8, 2017, at 4:34 PM, Jean-Paul Calderone <exarkun at twistedmatrix.com>
> wrote:
>
>
> Here's one example I know of off the top of my head, <
> https://github.com/ScatterHQ/flocker/blob/master/flocker/
> restapi/testtools.py#L316>.  This one isn't a from-scratch
> re-implementation of the implicit Request interface but a Request subclass,
> instead.  However, it still interacts with a lot of important pieces of
> Request which aren't part of IRequest.
>
>
> Mark already addressed how he won't be breaking this use-case (which is
> hugely important, and core to the whole idea of a compatibility policy, so
> that is as it should be).
>
> However, this kind of test-mocking is, I think, ultimately done at the
> wrong layer.  It's trying to override some very vaguely-specified internals
> in the middle of an implementation.
>
>
Absolutely.



> Really, twisted.web should provide its own testing tools, of course.  But
> if you're going to implement something that does this sort of overriding, I
> think the idiom to follow would be treq.testing: https://github.
> com/twisted/treq/blob/fcf5deb976c955ca6ef6484f414d25
> 839932940e/src/treq/testing.py, rather than any of the various
> implementations of DummyRequest (including more than a few I'm sure I've
> written).
>

Though, note, the link I gave above was support code for something very
similar to this treq code:
https://github.com/ScatterHQ/flocker/blob/master/flocker/restapi/testtools.py#L460

To clarify your point a bit (I think):


   - MemoryAgent (from Flocker) provides a testing IAgent by implementing
   an IRequest that does in-memory resource traversal to dispatch requests and
   generate responses.
   - RequestTraversalAgent (from treq) provides a testing IAgent by
   implementing (using) iosim to do in-memory protocol/transport interacts to
   drive an in-memory HTTP conversation that runs all of the regular Twisted
   Web HTTP processing machinery.

RequestTraversalAgent's approach is better because the protocol/transport
interface is better defined.  Because it's better
defined, RequestTraversalAgent doesn't have to touch pieces that we might
want to consider implementation details (whether they're _-prefixed or
not).  It also invokes a larger portion of the real implementation code
making it more likely to be a realistic simulation of real-world use of the
code.

Having spelled this out, what occurs to me now is
that RequestTraversalAgent is really just a step or so up the ladder and
there's further to go.  For example, RequestTraversalAgent only needs to
exist at all because `iosim` has a distinct interface.  This distinct
interface means you need a RequestTraversalAgent-like thing for each
reactor-using thing for which you want to provide a test double.  If this
adaption behavior were bundled up differently then I think we'd get a lot
more in-memory test doubles for free (or closer to it - we'd be another
rung up the ladder, at least).  That seems like it would be a big win given
how *few* of these testing helpers Twisted has historically managed to
provide (suggesting it's just too hard to do so given the current tools).


>
> This is a set of ideas that I've been gradually arriving at over a long
> period of time, but it's probably high time for some public discussion by
> now, even if it's just everybody saying "yeah, that sounds good" :-).
>

So, it pretty much sounds good, though with the above refinements. :)


>
> On a related note, 'proto_helpers' is in a super awkward place.
>  'twisted.testing', anyone? :)
>

Yes.  I almost suggested this about a week ago when I was preparing to
contribute some testing code but then realized I couldn't contribute the
code after all. :(

Jean-Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20170109/625c5099/attachment-0002.html>