[Twisted-Python] buildbot/osx10.10-py2.7 and required builders for merge

[Glyph]

> On Jul 5, 2017, at 2:29 AM, Adi Roiban <adi at roiban.ro> wrote:
> 
> Hi,
> 
> I did not had to much time to follow twisted-infra activity lately,
> sorry if the following question is obvious for all the others.
> 
> Why the OSX build was removed from Travis and forced on buildbot?

As Craig said: it was slow.

Of course, the entire CI process is slow, so just a little more slowness would not have been a reason.  It wasn't just slow, it was catastrophically slow: sometimes blocking builds for entire days at a time.

Not only was it an order of magnitude out from all other build slowness, it would also block other travis builds from completing promptly.

And finally, the frequent outages on Travis's infrastructure meant that not only was it slow, not only was it interfering with other resources, but it was also tremendously unpredictable: so sometimes it would be only a few minutes, sometimes 72 hours. Intermittent reinforcement is a form of psychological torture, so this was really the icing on this terrible cake.

> I think that for contributor without write access to the main repo,
> this is a big handicap as they can't get a good coverage report ...
> and merging such branches needs extra work as you need to create a new
> branch, push it wait for results...(maybe create a new PR)... etc

Yep, it's really annoying and I very much wish we could turn it back on.  What it comes down to is: would it be more useful for the contributors to get their Linux / Py2 / Py3 build results immediately automatically, or their macOS build results after 15 hours automatically?

But, loading https://www.traviscistatus.com <https://www.traviscistatus.com/> today, I see yet another effective outage on the OS X infra:



The frequency of outages that merit a formal incident report does seem to have gone down, but "you can't do a build for the entire working day because the infrastructure is regularly jammed up" is a level of availability that is more like a continuous outage :).

> Is the list of builders marked in GitHub as "required" the official
> list of tests which need to be green in order to merge?

Technically speaking all builders should pass.  Particularly, patch coverage in principle ought to be enforced.

However:

In some cases (such as trivial python 3 syntax transformations) there are patches which should be able to skip the coverage check, so it isn't enforced.  It would be better if these were all covered, but sometimes it makes more sense to get a module importing on py3 so that we can address coverage issues more easily.
Also, codecov itself is unreliable enough that making this mandatory would make us spend more time investigating issues with codecov than developing twisted.  (And other options, such as coveralls, seem to just be worse.)
Given the large number of builders in our fleet, it's likely that at least one will still fail with some intermittent issue that's irrelevant to the patch.  Until we have less activity on https://twistedmatrix.com/trac/ticket/8879 <https://twistedmatrix.com/trac/ticket/8879>, making these mechanically mandatory would still do more harm than good.  However… we have been making a lot of progress, and we certainly see a lot fewer intermittent failures than we used to!

> I am reading this page, but is not clear which tests need to be green:
> https://twistedmatrix.com/trac/wiki/ReviewProcess#Authors:Howtomergethechangetotrunk <https://twistedmatrix.com/trac/wiki/ReviewProcess#Authors:Howtomergethechangetotrunk>

Yeah, the documentation on this is a lot spottier than it should be.  Please feel free to propose some changes to that text based on this.  Thanks for raising this issue!

-g

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20170705/4295f524/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-1.png
Type: image/png
Size: 30815 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20170705/4295f524/attachment-0002.png>