[Twisted-Python] Testing Twisted code without trial

[exarkun at twistedmatrix.com]

On 18 Mar, 07:32 pm, adi at roiban.ro wrote:
>On 22 January 2013 22:03,  <exarkun at twistedmatrix.com> wrote:
>>On 09:29 am, adi at roiban.ro wrote:
>>>On 22 January 2013 02:21,  <exarkun at twistedmatrix.com> wrote:
>>>>On 20 Jan, 02:35 pm, adi at roiban.ro wrote:
>
>I agree that this is a ugly hack and I removed the project.
>>
>>Hi Adi,
>>
>>trial does what it does by touching a lot of internal stuff as well.
>>This is still bad, but at least it's our fault if it ever breaks 
>>instead
>>of yours.  There's also a long term plan (or "plan" may be putting it
>>too strongly, perhaps I should say "hope") that this part of trial 
>>will
>>change to only use public interfaces.  This will probably require
>>reactors actually implementing restartability, or it will require
>>changing the trial feature slightly (eg, so it starts a reactor, runs
>>all tests, then stops the reactor - if it did this, I'm sure you can
>>imagine how "waiting" for a Deferred would just be adding a callback 
>>to
>>the right place, as in any other Twisted-based application).
>>
>>Are you interested in helping out with making reactors restartable? :)
>
>Sorry for the late reply.
>
>I am still clumsy when working with Twisted so I don't know if I can
>help to much here.
>
>I don't know what is expected from a restartable reactor.
>
>The way I am testing deferreds is by starting the reactor, allow for
>the deferred to execute and then stop the reactor.
>
>I don't want to pause it and then continue the execution from where it
>was stopped.
>
>To help with debugging I am also printing a snapshot of reactor state
>at a certain time.
>
>----
>
>I prefer the Arrange/Act/Assert way of writing test:
>
>
>checker = mk.credentialsChecker()
>credentials = mk.credentials()
>
>deferred = checker.requestAvatarId(credentials)
>failure = self.getDeferredFailure(deferred)
>
>self.assertFailureType(AuthentiationError, failure)
>self.assertEqual(credentials.username, failure.value.username)
>
>
>I found it easier to read than this version:
>
>
>checker = mk.credentialsChecker()
>credentials = mk.credentials()
>
>    def check_result(result_or_failure):
>        self.assertFailureType(AuthentiationError, failure)
>        self.assertEqual(credentials.username, failure.value.username)
>
>deferred = checker.requestAvatarId(credentials)
>deferred.addBoth(check_result)
>
>return deferred
>
>--------
>
>I have updated the code to use as many public reactor members as 
>possible.
>
>The following private member are still use:
>reactor._startedBefore, reactor._started
>
>It uses the following public methods:
>startRunning(), doIteration(), stop(), iterate()
>
>Here is the main part that blocks the execution until the deferred got 
>a result.
>It executes the deferred in the reactor loop.
>
>https://github.com/chevah/empirical/blob/master/chevah/empirical/testcase.py#L240
>
>------
>
>Maybe this is has only limited usage, but I just wanted to share this 
>work.
>For me, this makes writing test a much nicer experience.

Hi Adi,

This basically looks like an implementation of the old, now-removed 
`TestCase.wait` API.

We got rid of `wait` for several reasons:

  * It was hard to implement.  By the end, it sort of worked with most 
reactors - but not all of them.

  * It is a tool for building non-deterministic, slow tests.  If tests 
are written *not* to do real I/O and *not* to wait for real time to 
pass, then they don't need to let a real reactor spin.

We replaced these ideas:

  * with returning a `Deferred` from a test method (which works even if 
you don't use trial to run your tests - but not if you don't subclass 
trial's `TestCase`).  We eventually moved on from this idea, though many 
parts of Twisted itself are still tested using this feature, to...

  * things like `twisted.test.proto_helpers.MemoryReactor`, 
`twisted.internet.task.Clock`, and most recently 
`TestCase.successResultOf` and `TestCase.failureResultOf` (but don't 
confuse these with your `getDeferredFailure` - they are significantly 
less capable).

I'd encourage you to explore testing strategies that use 
reactor/transport/time fakes and give us feedback about where they're 
not making your job easy enough.  I think ultimately you'll be happier 
with the resulting tests, and you won't have to maintain so much hairy 
reactor manipulation code.

Jean-Paul