[Twisted-Python] Problem with trial using the reactor
jml at mumak.net
Thu Jun 7 00:58:28 EDT 2007
On 6/7/07, Alex Lang <alex.lang at mail.mcgill.ca> wrote:
> Here is a problem regarding the distributed trial test runner. The
> slave instances receive data from the master, telling them which tests
> to run. This requires use of the reactor on the slave-side.
> Unfortunately, when the testCase does the cleanup of the reactor, it
> cleans up everything including the trial slave.
> Any ideas on what should be done about this?
This is a roundabout way of answering your question, but it's helpful
for me to write it down, and I think it will inform the discussion.
In terms of Trial's requirements:
* We want to write and run tests that perform asynchronous operations,
which essentially means tests that return Deferreds.
* We want to be able to run these tests in non-Trial test runners.
* We want tests to be isolated as much as possible.
* We want to be able to use Trial to test the reactor itself.
Some awkward facts:
* There is exactly one reactor per Twisted process. This reactor
cannot be cleanly restarted in a cross-platform way.
* The current Twisted test suite (i.e. what is loaded when you do
'trial twisted') does not play nicely with the reactor. Changing this
will take a significant amount of work.
So, in response to all of these things:
* Trial presents a blocking interface for running any given test:
TestCase.run(). This allows Deferred-returning tests to be used in
non-Twisted test runners.
* By design, Trial totally obliterates the reactor in each call to
TestCase.run(). This helps ensure test isolation
* Trial's design operates on the assumption that the caller of
TestCase.run() will not be doing anything with the reactor.
However, all of this presents problems:
* The vast bulk of Twisted's code base cannot be used to implement a
Trial runner. This is what Alex is discovering now.
* Trial's cleanup code is buggy and fragile. Changing anything is
likely to break hundreds of tests.
* Tests that use twisted.trial.unittest.TestCase run in a
highly-specialized, somewhat unrealistic environment.
Various solutions have been proposed:
* Using a customized reactor wrapper in trial that tracks resources
created by tests and cleans up only those resources.
* Making reactors restartable.
* Allowing for multiple running reactors. i.e. A reactor that could be
started within another running reactor.
I believe that only the third of these solutions would actually help
solve Alex's problem. However, as I understand it, the solution would
require extensive API changes across most of Twisted, as things which
once assumed a global reactor must now take 'reactor' as a parameter.
I'd rather not offer any recommendations in this email. It's difficult
enough figuring out what the problem is and what the options are.
 For example, Trial does two calls to 'reactor.iterate()' in its
cleanup. If these calls are removed, tests fail and there are a
significant number of cleanup errors. I discussed some of the issues
here in an email last year.
More information about the Twisted-Python