[Twisted-Python] Running Twisted tests with testtools

[Jonathan Lange]

On Wed, Feb 8, 2012 at 5:22 PM, Free Ekanayaka <free at 64studio.com> wrote:
> Hi Glyph,
>
> |--==> On Wed, 8 Feb 2012 02:05:22 -0500, Glyph <glyph at twistedmatrix.com> said:
>
...
>  G> It's already compatible, as Jonathan said.  You can just use the
>  G> trial runner with testtools tests.
>
> Even those returning deferreds and using AsynchronousDeferredRunTest? I
> presume so, just asking to be sure.
>

Yes.

(Also, you don't get sure from asking, you get sure from trying).

> Also, testtools has an AsynchronousDeferredRunTestForBrokenTwisted class
> too which I believe mimics more closely what trial currently does, so
> I'm not sure what the difference between AsynchronousDeferredRunTest and
> AsynchronousDeferredRunTestForBrokenTwisted would be when using trial to
> run such tests.
>

>From the documentation:

* Trial spins the reactor a couple of times before cleaning it up,
  ``AsynchronousDeferredRunTest`` does not.  If you rely on this behavior, use
  ``AsynchronousDeferredRunTestForBrokenTwisted``.

And the API documentation:

    """Test runner that works around Twisted brokenness re reactor junk.

    There are many APIs within Twisted itself where a Deferred fires but
    leaves cleanup work scheduled for the reactor to do.  Arguably, many of
    these are bugs.  This runner iterates the reactor event loop a number of
    times after every test, in order to shake out these buggy-but-commonplace
    events.
    """

>  >>However it'd be good if Twisted took a clear position on this,
>
>  G> Nope.  We write software, not position papers :).
>
> Okay maybe I used the wrong wording. I was asking for a "position"
> because I got the impression that Jonathan is considering testtools a
> sort of trial2 that provides improvements over trial, but that was not
> likely going to replace only it because of compatibility reasons. Please
> Jonathan correct me if it's not the case.
>
> So *if* testtools is considered superior to trial for some reason, but
> it's problematic for Twisted to internally drop trial in favor of
> testtools, I was wondering if it was the case to (also) point people to
> testtools from the Twisted website/documentaiton, or something along
> those lines.
>

The passive voice here is masking an ambiguity. I consider testtools
superior to trial in many ways. Some others disagree, or don't have
enough information to form an opinion.

Perhaps it's worth a mention in the Twisted documentation, but I
personally won't push too hard for that.

...
>  G> I think that the main risks to "languishing" or "not taking off"
>  G> for both Trial and testtools are issues with their own
>  G> documentation and promotional materials, not in any conflict with
>  G> each other.  If you're worried about it, contribute code or
>  G> documentation or blog-posts or what have you to one or both
>  G> projects :).
>
> Probably a good addition to the documentation of the two projects would
> be to describe the differences between the two and if there is any
> reason one should use one or the other, beside bare taste. I can surely
> help with that when I have better picture myself.
>

For testtools, we'd also have to do that for nose, nose2, unittest2,
zope.testing and py.test for it to make any sense. Heck, just listing
all of those makes me regret ever writing any test framework code in
the first place.

I'm not sure I'd want that documentation, to be honest. I would only
ever do work towards it if it meant that one test framework could be
killed, even if that ended up being testtools.

cheers,
jml