[Twisted-Python] How to say "reverted". (was Re: [Twisted-commits] r22628 - Revert r22624: regression in test_conch.)

[Andrew Bennetts]

Thomas Hervé wrote:
[...]
> > [1] I'll be even happier when we have *machines* monitoring the test
> > suite, but that's another email.
> 
> Currently, our test suite is a little bit too unreliable to do this 
> (aka 'intermittent failures'). But maybe in a near future...

We've had intermittent failures for *years*.  I doubt that'll get solved in a
near future without some sort of significant change to how we do things.

Here's a thought experiment: one possibility would be completely automate the
running of the test suite, as jml is suggesting, e.g. by using a commit hook (or
a “commit bot” like PQM) that will run the test suite before allowing a commit
to trunk.  Then intermittent failures become much more intolerable, because they
will regularly frustrate developers rather than being something you can usually
ignore.  Thus people will be much more motivated to find solutions to our
intermittent tests (whether by fixing the offending tests, or disabling them, or
something else).

I'm not sure it'd be worth the effort, but it's interesting to think about...

-Andrew.