[Twisted-Python] Thoughts about testing

Antony Kummel antonykummel at yahoo.com
Tue Oct 25 06:19:43 MDT 2005


Hi all,

I just want to share some ideas that I have had.

>From my experience, testing with Trial is too hard. My
problems can be divided into three categories: 1.
obscure error reporting, 2. unclean reactor 3. whoop
whoop whoop. 

I am also pretty sure that I'm doing things that Trial
does not want me to do, such as actually opening
sockets and communication over the network. I'm saying
this because it does not seem to happen in the Twisted
test-suite. But I don't think there is a good reason
for this limitation.

Actually, in sum, I think there are too many
limitations on writing tests with Trial, and they all
boil down to the unclean reactor problem (or at least
most of them). I want to suggest an alternative
approach.

How about not requiring the reactor to be clean at the
end of a test? If anyone wants to make sure that
anything they do leaves the reactor in a clean state,
they can test for it themselves. This seems to me more
like a constraint imposed by the implementation of
Trial rather than a useful feature. Also, parts of
Twisted itself are practically unusable inside a test
because they leave the reactor dirty (such as threaded
address resolution). An alternative feature could be
enabling the user to specify that a certain delayed
call or thread is allowed to remain after the test,
and then Trial won't complain. The only question
remaining is how to do it. Simple: use a different
process. Run the tests in a different process, and
create a new one each time the reactor is dirtied.
py.execnet is a nice example of this concept.

The second thought is this: there seem to be popping
up different testing toolkits each with their own very
nice extensions and features
(http://testoob.sourceforge.net/,
http://codespeak.net/py/current/doc/test.html). Trial
cannot benefit from this, having branched away at the
pyUnit level. I think Trial's special features can be
relatively easily formulated as plugins to a
plugin-oriented testing framework (especially if the
clean reactor requirement is relieved), and so can the
other testing packages. What this means, is that the
best thing anyone who wants the world of unit testing
to develop, and to benefit from it, is to push for a
plugin-oriented pyUnit, and for an implementation of
Trial (and the other tools) as a plugin for that
framework. I think.

Any comments?

Antony Kummel


		
__________________________________ 
Yahoo! FareChase: Search multiple travel sites in one click.
http://farechase.yahoo.com




More information about the Twisted-Python mailing list