[Twisted-Python] probably stupid question about automated testing and reactor.run

[Jean-Paul Calderone]

`reactor` is global mutable state.  So all of the usual tragedies of such a
thing apply when you consider how unit tests will interact with it.

Twisted has some APIs that will jump through some hoops for you to try to
mitigate the damage done by this early design mistake.  For example,
there's a context manager that will swap out the current reactor with a new
object specified by you.  *If* the code under test doesn't import the
reactor until it is invoked, this can help (this is just one reason a lot
of modern Twisted code avoids importing the reactor until it absolutely
needs it - of course, another thing modern Twisted code often does is
accept the reactor as an argument instead of importing it and if you do
this then you can escape some of the "global" part of the problem).

But you still may not want to have `reactor.run` in your application code.
There are other problems - such as the fact that it's common for the
*failure* mode of tests for such code to be to hang indefinitely.  Or for
test failures to be obscure in other ways due to the fact that you're
invoking pretty much your *whole* stack at once - not so much a "unit" test
- and so finding the cause of the problem is a challenge. Or for tests to
work on one run but fail on another due to non-determinism in event
delivery/dispatch.  Or for tests to work on one platform but fail on
another for similar reasons.

So, think about the goals you have for the tests you want to write.  You
may indeed want *some* tests for reactor.run-calling code but my guess is
that you want the majority to be for "pure" (in the functional sense) code
that stays pretty far from the reactor.  Of course, if you're using a
Twisted-based library that hasn't thought out the testing story for
application code, you may have some roadblocks in your way.

Personally, the most recent Twisted-using application I wrote, I factored
all of my "start up Twisted and configure the process and such" code into
one re-usable component and manually verified it works, then wrote
automated tests (mostly unit) for everything *else*.  In some ways, this is
what `IService` gives you, though if you find yourself wanting to go beyond
what it offers you may have to write something like what I did.

Finally, Twisted has a bunch of blackbox tests *for reactor
implementations* which
essentially must call reactor.run.  It's not clear how applicable the
strategies used by these tests might be to tests for application code
(rather than reactor implementation code) but it might be worth looking
at.  These tests are mostly in twisted/internet/test/ - eg test_tcp.py in
that directory.

Jean-Paul

On Wed, Nov 7, 2018 at 11:14 AM Chris Withers <chris at withers.org> wrote:

> Hi All,
>
> Sorry, the silly questions about testing keep coming...
>
> Is it legit for a unit testable piece of an application's code to call
> reactor.run()?
>
> How do you feed a reactor.stop() into such a test? Would that be a sane
> thing to try and do?
>
> How do people deal with this kind of testing?
>
> cheers,
>
> Chris
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20181107/9363059c/attachment-0002.html>