[Twisted-Python] How to say "reverted". (was Re: [Twisted-commits] r22628 - Revert r22624: regression in test_conch.)

Thomas Hervé therve at free.fr
Thu Feb 21 05:45:46 EST 2008


Quoting Andrew Bennetts <andrew-twisted at puzzling.org>:

> Thomas Hervé wrote:
> [...]
>> > [1] I'll be even happier when we have *machines* monitoring the test
>> > suite, but that's another email.
>>
>> Currently, our test suite is a little bit too unreliable to do this
>> (aka 'intermittent failures'). But maybe in a near future...
>
> We've had intermittent failures for *years*.  I doubt that'll get solved in a
> near future without some sort of significant change to how we do things.
>
> Here's a thought experiment: one possibility would be completely automate the
> running of the test suite, as jml is suggesting, e.g. by using a   
> commit hook (or
> a ?commit bot? like PQM) that will run the test suite before   
> allowing a commit
> to trunk.  Then intermittent failures become much more intolerable,   
> because they
> will regularly frustrate developers rather than being something you   
> can usually
> ignore.  Thus people will be much more motivated to find solutions to our
> intermittent tests (whether by fixing the offending tests, or   
> disabling them, or
> something else).
>
> I'm not sure it'd be worth the effort, but it's interesting to think about...


I think it definitely worth the effort. As I see it, maybe we could  
split the effort in 2 parts:
  * having one commit bot for our best supported platform. I think  
debian-py2.4-select has been free of intermittent failure for a bit  
now. Maybe we could trust it enough to disallow commits that breaks  
it. The other advantage for this is that we could formalize the merge  
process a bit more: for example, with a web page where you specify the  
branch name, authors, reviewers, tickets fixed, etc. It looks like  
what PQM is doing.
  * having better notifications for trunk commits that generate failed  
runs. Today, this work is basically done by JP looking at the buildbot  
and remembering the failures and tickets linked to them, and possibly  
creating a new ticket if it's not an already identified problem. It's  
not only cumbersome but probably let some errors pass through.


We could look at this after the release :).

-- 
Thomas






More information about the Twisted-Python mailing list