[Twisted-Python] Testing Strategies (was Re: Streaming Requests)

[Glyph Lefkowitz]

> On Jan 12, 2017, at 4:24 AM, Jean-Paul Calderone <exarkun at twistedmatrix.com> wrote:
> 
> On Wed, Jan 11, 2017 at 11:41 PM, Glyph Lefkowitz <glyph at twistedmatrix.com <mailto:glyph at twistedmatrix.com>> wrote:
> 
>> On Jan 9, 2017, at 4:13 AM, Jean-Paul Calderone <exarkun at twistedmatrix.com <mailto:exarkun at twistedmatrix.com>> wrote:
>> 
>> On Mon, Jan 9, 2017 at 4:52 AM, Glyph Lefkowitz <glyph at twistedmatrix.com <mailto:glyph at twistedmatrix.com>> wrote:
>> On a related note, 'proto_helpers' is in a super awkward place.  'twisted.testing', anyone? :)
>> 
>> Yes.  I almost suggested this about a week ago when I was preparing to contribute some testing code but then realized I couldn't contribute the code after all. :(
> 
> Well, now it's just a matter of time :).
> 
> Some further thoughts on this... What I've actually been doing elsewhere (mostly) is to put the testing implementations quite near the real implementations.  For example, I recently introduced `workproject.subscription_manager.{network_client,memory_client}`.  But contrary to this, in txAWS I'm following the pre-existing pattern which is more like `txaws.service.AWSServiceRegion.get_ec2_client` & `txaws.testing.service.FakeAWSServiceRegion.get_ec2_client`.

"Memory" implementations are definitely a step up the value hierarchy from "fake" implementations.

I guess maybe both of these patterns make sense for certain things.  It makes sense to put a memory implementation next to the real one (c.f. SQLite), but it doesn't make as much sense to put a testing implementation there.  I would make this distinction by saying a "testing" implementation is one which has assertion method helpers, simulated I/O triggers, and similar.

> Which I prefer, I'm not really sure yet.  Role-named modules are easy to recognize.  On the other hand, testing code next to implementation code is easier to stumble across and you don't have to recognize a separate testing module to find it.  Also, role-named modules tend to pile up and get in each other's way (start working with `twisted.testing`, `txaws.testing`, etc and things start to get confusing quickly).

Yeah, and I guess it'll actually be `twisted.internet.testing`, `twisted.python.testing`, `twisted.web.testing` etc.  Which we explicitly rejected with `interfaces` because it got too confusing; the existence of multiple `endpoints` modules has been mildly irritating while working on HostnameEndpoint for the last couple of days.

> Additionally, a related idea is that often the testing implementation might actually serve as a good starting place for the real implementation.  I think the code in txAWS is a good example of this (ie, the real and fake implementations duplicate a lot of logic with some pointless divergences).  In the olden days I would have said "the testing implementation is often a good base class for the real implementation".  But now I'll mumble something about composition.

Yeah, I think this is the broader point, actually - the best possible "fake" implementation is a completely real implementation where the lower level it's composing against is fake.

> Put another way:
> Clock should be the real implementation of all of the time-source independent logic of IReactorTime
One minor tweak: Clock itself should be split into two objects, one of which has just 'advance' and one of which has IReactorTime <https://glyph.twistedmatrix.com/2015/05/separate-your-fakes-and-your-inspectors.html <https://glyph.twistedmatrix.com/2015/05/separate-your-fakes-and-your-inspectors.html>>.  Possibly three objects, since it gets its `seconds` from somewhere else?
> FakeAWSServiceRegion should be the real implementation of the "client factory" logic (get_XXX_client) of AWSServiceRegion 
> and so on.

Yes, definitely.

> Perhaps this also speaks to the idea you mentioned that there should be a stack of thin abstractions but I snipped that content already so I'll try responding to that separately, later.


Ideally, the only "fake" implementation we should need is StringTransport and Clock; the abstractions in the middle would be wiring together the layers in a way that's convenient for test clients to access.

-glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20170112/2b17487a/attachment-0002.html>