[Twisted-Python] Testing Strategies (was Re: Streaming Requests)

[Glyph Lefkowitz]

On Jan 8, 2017, at 4:34 PM, Jean-Paul Calderone <exarkun at twistedmatrix.com> wrote:
> 
> Here's one example I know of off the top of my head, <https://github.com/ScatterHQ/flocker/blob/master/flocker/restapi/testtools.py#L316 <https://github.com/ScatterHQ/flocker/blob/master/flocker/restapi/testtools.py#L316>>.  This one isn't a from-scratch re-implementation of the implicit Request interface but a Request subclass, instead.  However, it still interacts with a lot of important pieces of Request which aren't part of IRequest.

Mark already addressed how he won't be breaking this use-case (which is hugely important, and core to the whole idea of a compatibility policy, so that is as it should be).

However, this kind of test-mocking is, I think, ultimately done at the wrong layer.  It's trying to override some very vaguely-specified internals in the middle of an implementation.

Really, twisted.web should provide its own testing tools, of course.  But if you're going to implement something that does this sort of overriding, I think the idiom to follow would be treq.testing: https://github.com/twisted/treq/blob/fcf5deb976c955ca6ef6484f414d25839932940e/src/treq/testing.py <https://github.com/twisted/treq/blob/fcf5deb976c955ca6ef6484f414d25839932940e/src/treq/testing.py>, rather than any of the various implementations of DummyRequest (including more than a few I'm sure I've written).

Internally, treq.testing uses MemoryReactor and the (not-technically-public-but-it-really-should-be) iosim module.  Therefore the layer it's plugging in at is at the very well-defined interface between IAgent and IStreamClientEndpoint, and between HTTPChannel and Transport.  This gives the determinism and (most of the) speed of an in-memory implementation, but a great deal of the realism of a full-blown integration test.  In particular, it binds to the appropriate objects within Twisted and will throw deprecation warnings if their interfaces change.  (Also of note; since what it's building a fake for is really 99% Agent, the bulk of this could probably move up another level into Twisted with very little effort.)

I think following this idiom of pushing I/O through a fake reactor and having a back-end that can be controlled via test-specific APIs is probably what we should all be striving for, and in Twisted we should be working to make the memory-reactor stuff more complete and convenient, rather than adding dummy implementations at each layer of the stack.

This is a set of ideas that I've been gradually arriving at over a long period of time, but it's probably high time for some public discussion by now, even if it's just everybody saying "yeah, that sounds good" :-).

On a related note, 'proto_helpers' is in a super awkward place.  'twisted.testing', anyone? :)

-glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20170109/9557646e/attachment-0002.html>