[Twisted-Python] How to say "reverted". (was Re: [Twisted-commits] r22628 - Revert r22624: regression in test_conch.)
therve at free.fr
Thu Feb 21 05:45:46 EST 2008
Quoting Andrew Bennetts <andrew-twisted at puzzling.org>:
> Thomas Hervé wrote:
>> >  I'll be even happier when we have *machines* monitoring the test
>> > suite, but that's another email.
>> Currently, our test suite is a little bit too unreliable to do this
>> (aka 'intermittent failures'). But maybe in a near future...
> We've had intermittent failures for *years*. I doubt that'll get solved in a
> near future without some sort of significant change to how we do things.
> Here's a thought experiment: one possibility would be completely automate the
> running of the test suite, as jml is suggesting, e.g. by using a
> commit hook (or
> a ?commit bot? like PQM) that will run the test suite before
> allowing a commit
> to trunk. Then intermittent failures become much more intolerable,
> because they
> will regularly frustrate developers rather than being something you
> can usually
> ignore. Thus people will be much more motivated to find solutions to our
> intermittent tests (whether by fixing the offending tests, or
> disabling them, or
> something else).
> I'm not sure it'd be worth the effort, but it's interesting to think about...
I think it definitely worth the effort. As I see it, maybe we could
split the effort in 2 parts:
* having one commit bot for our best supported platform. I think
debian-py2.4-select has been free of intermittent failure for a bit
now. Maybe we could trust it enough to disallow commits that breaks
it. The other advantage for this is that we could formalize the merge
process a bit more: for example, with a web page where you specify the
branch name, authors, reviewers, tickets fixed, etc. It looks like
what PQM is doing.
* having better notifications for trunk commits that generate failed
runs. Today, this work is basically done by JP looking at the buildbot
and remembering the failures and tickets linked to them, and possibly
creating a new ticket if it's not an already identified problem. It's
not only cumbersome but probably let some errors pass through.
We could look at this after the release :).
More information about the Twisted-Python