[Twisted-Python] Testing Twisted code without trial

Adi Roiban adi at roiban.ro
Tue Mar 26 06:39:25 EDT 2013


On 25 March 2013 15:48,  <exarkun at twistedmatrix.com> wrote:
> On 18 Mar, 07:32 pm, adi at roiban.ro wrote:
>>On 22 January 2013 22:03,  <exarkun at twistedmatrix.com> wrote:
>>>On 09:29 am, adi at roiban.ro wrote:
>>>>On 22 January 2013 02:21,  <exarkun at twistedmatrix.com> wrote:
>>>>>On 20 Jan, 02:35 pm, adi at roiban.ro wrote:
>>
>>I agree that this is a ugly hack and I removed the project.
>>>
>>>Hi Adi,
>>>
>>>trial does what it does by touching a lot of internal stuff as well.
>>>This is still bad, but at least it's our fault if it ever breaks
>>>instead
>>>of yours.  There's also a long term plan (or "plan" may be putting it
>>>too strongly, perhaps I should say "hope") that this part of trial
>>>will
>>>change to only use public interfaces.  This will probably require
>>>reactors actually implementing restartability, or it will require
>>>changing the trial feature slightly (eg, so it starts a reactor, runs
>>>all tests, then stops the reactor - if it did this, I'm sure you can
>>>imagine how "waiting" for a Deferred would just be adding a callback
>>>to
>>>the right place, as in any other Twisted-based application).
>>>
>>>Are you interested in helping out with making reactors restartable? :)
>>
>>Sorry for the late reply.
>>
>>I am still clumsy when working with Twisted so I don't know if I can
>>help to much here.
>>
>>I don't know what is expected from a restartable reactor.
>>
>>The way I am testing deferreds is by starting the reactor, allow for
>>the deferred to execute and then stop the reactor.
>>
>>I don't want to pause it and then continue the execution from where it
>>was stopped.
>>
>>To help with debugging I am also printing a snapshot of reactor state
>>at a certain time.
>>
>>----
>>
>>I prefer the Arrange/Act/Assert way of writing test:
>>
>>
>>checker = mk.credentialsChecker()
>>credentials = mk.credentials()
>>
>>deferred = checker.requestAvatarId(credentials)
>>failure = self.getDeferredFailure(deferred)
>>
>>self.assertFailureType(AuthentiationError, failure)
>>self.assertEqual(credentials.username, failure.value.username)
>>
>>
>>I found it easier to read than this version:
>>
>>
>>checker = mk.credentialsChecker()
>>credentials = mk.credentials()
>>
>>    def check_result(result_or_failure):
>>        self.assertFailureType(AuthentiationError, failure)
>>        self.assertEqual(credentials.username, failure.value.username)
>>
>>deferred = checker.requestAvatarId(credentials)
>>deferred.addBoth(check_result)
>>
>>return deferred
>>
>>--------
>>
>>I have updated the code to use as many public reactor members as
>>possible.
>>
>>The following private member are still use:
>>reactor._startedBefore, reactor._started
>>
>>It uses the following public methods:
>>startRunning(), doIteration(), stop(), iterate()
>>
>>Here is the main part that blocks the execution until the deferred got
>>a result.
>>It executes the deferred in the reactor loop.
>>
>>https://github.com/chevah/empirical/blob/master/chevah/empirical/testcase.py#L240
>>
>>------
>>
>>Maybe this is has only limited usage, but I just wanted to share this
>>work.
>>For me, this makes writing test a much nicer experience.
>
> Hi Adi,
>
> This basically looks like an implementation of the old, now-removed
> `TestCase.wait` API.
>
> We got rid of `wait` for several reasons:
>
>   * It was hard to implement.  By the end, it sort of worked with most
> reactors - but not all of them.
>
>   * It is a tool for building non-deterministic, slow tests.  If tests
> are written *not* to do real I/O and *not* to wait for real time to
> pass, then they don't need to let a real reactor spin.
>
> We replaced these ideas:
>
>   * with returning a `Deferred` from a test method (which works even if
> you don't use trial to run your tests - but not if you don't subclass
> trial's `TestCase`).  We eventually moved on from this idea, though many
> parts of Twisted itself are still tested using this feature, to...
>
>   * things like `twisted.test.proto_helpers.MemoryReactor`,
> `twisted.internet.task.Clock`, and most recently
> `TestCase.successResultOf` and `TestCase.failureResultOf` (but don't
> confuse these with your `getDeferredFailure` - they are significantly
> less capable).
>
> I'd encourage you to explore testing strategies that use
> reactor/transport/time fakes and give us feedback about where they're
> not making your job easy enough.  I think ultimately you'll be happier
> with the resulting tests, and you won't have to maintain so much hairy
> reactor manipulation code.

I already use StringTransport in various forms for unit and
integration tests and it is great!
Clock is also great for delayedCalls. I also have various
mock/dummy/spy implementations for transports/channels.

The whole Twisted architecture make writing tests a fun task! Thanks!

I am "spining" the reactor to "resolve" all deferreds involved in a
StringTransport conversation or in an DeferredList or other kind of
chained deferreds. These are what I call "integration tests" and they
only use memory, no external I/O.

In most of my calles of result = self.getDeferredResult(deferred), the
deferred's callback() method was already called and I just want to
resolve the callbacks chain.

Is there something in _synctest that can "resolve" a list/chain of deferreds?

I was not aware of MemoryReactor. Thanks for the note. From what I can
read in the code, it does not help with spinning the reactor.

-----


I only do real I/O for what I call, "system tests" these are checking
integration of my code with the outside world (the system). They are
kept to a minimum.
Since there should not be to many of these tests, returning a deferred
is not a big annoyance... just that the code is harder to read.

Thanks again for all your feedback!
-- 
Adi Roiban



More information about the Twisted-Python mailing list