[Twisted-Python] Re: Refactoring of trial - call for feature requests and suggestions

David Bolen db3l at fitlinxx.com
Wed Jun 16 14:34:31 EDT 2004

Glyph Lefkowitz <glyph at divmod.com> writes:

> There are multiple, parallel issues here :).  First is that the reactor
> needs a genuine make-sure-i'm-cleaned-up function.  The current usage of
> iterate() has its roots in the fact that shutdown is (used to be?)
> horribly buggy and slightly destructive - if memory serves, the original
> problem was that the threadpool was destroyed and could not be
> re-created properly.

A general cleanup function would be much appreciated, as clearly our
biggest problem with tests involving the reactor has been the proper
cleanup.  While I can clean up pending delayed calls with an API call,
I've had to cheat and reach into the reactor to flush out pending
shutdown events during tests of modules that make use of them.

But to be honest, even better than a cleanup function would be the
ability to truly re-create/re-start the reactor.  It's always bothered
me that while everything else about the unit tests for my application
are fully independent (each test sets up and tears down everything it
needs), that the reactor is a persistence across tests.  It's led to a
few subtle issues in the tests that wouldn't have been there if each
test just created its own reactor.  Of course, this might be an ugly
change because of the way reactor instantiation right now is tied into
the import mechanism, rather than through an explicit API
initialization call.

> Second is that we shouldn't be waiting on deferreds in test code any
> more than we should be waiting on deferreds in real code.  The different
> approaches to waiting on deferreds currently mean there are still lots
> of places the tests can hang.  With a more unified approach, we could
> insert default timeouts on ALL tests more easily, and avoid loops that
> wait for multiple deferreds by using, e.g. DeferredList.

Coming from the perspective of someone that uses deferredResult/Error a lot
in testing my application (not twisted's own tests), I will say that they
have a very desirable characteristic of permitting tests to be written in
a more classic linear flow (no callbacks) while properly testing code that
is using deferreds under the covers.  Since tests by their nature tend to be
focused on a very specific and narrow corridor of behavior, it's pretty
rare not to be able to turn individual deferrable operations into blocking
ones while still testing precisely what you want to test.

I have a feeling that even if various reactor changes were made to
make it more test friendly, I'd still try to find some form of
mechanism that let me wait on individual deferrable operations to
provide the increased readability of the tests.  Of course, I wouldn't
care if that just means that deferredResult just becomes the wrapper
that delays until the callback happens, but doesn't have to use
iterate(), nor would I object to having a timeout on such blocking
wrappers, to avoid hanging a test.

But I do think that writing most of my tests with actual callbacks
would end up obfuscating them more than clarifying, without (IMHO) any
real significant improvement in the reality of what was being tested.

-- David

More information about the Twisted-Python mailing list