[Twisted-Python] GitHub Actions parallelism limit increase

[Adi Roiban]

On Tue, 6 Apr 2021 at 19:58, Kyle Altendorf <sda at fstab.net> wrote:

> On 2021-04-01 09:36, Adi Roiban wrote:
>
>
Thanks Kyle for the GitHub communication.

I have checked Twisted billing and we have GitHub Teams enabled.
So yes, 60x concurrent jobs on GitHub own VMs.

GitHub billing now shows $124.00 / month to cover our Team plan but we
don't have to pay.

For reference Twisted has 41 members [0]

Upgrading to enterprise will be  $903.00 / month :)

----------

We now use 13x jobs for a build - 10 - python + 3 lint/mypy/docs.
Python jobs now take a maximum of 8 minutes (on 2xCPU VMs with 4 parallel
jobs) [1] ... from the previous 30+ minutes

In the last month there were about 250 jobs so about 8 per day.

So I think that we are ok for now.

----------

We have 12 extra jobs on Azure.

I think that as an experiment we can move 1 Windows and 1 macOS job from
Azure to GitHub Actions.

I would still keep Azure pipelines jobs to use those extra VMs that are
available.
Azure allows retrying a single job on failure , or all failed jobs. This is
a bit nicery for flaky tests.
In GitHub, you have to re-run the whole workflow.

-----

We can get more jobs by hosting our own runners... but that is a different
story.

[snip]

Sounds good.  Sounds like exactly what I suggested.  :]  Except for
> maybe misunderstanding about what level of separation is needed for the
> interjob dependencies.
>
> Hopefully this clarifies a bit.  I think we actually intend the same
> thing.  Unless I've missed something else?
>

I guess that is best to discuss directly on the change made for a PR and
see how the final matrix will look.

Top priority for me is enabling trial parallel runs on Windows (PR waiting
for review) to use the extra CPU available on the VM.
Next is enabling some kind of PR triage helper.

[0] https://github.com/orgs/twisted/people
[1] https://github.com/twisted/twisted/actions/runs/735543893
[2]
https://dev.azure.com/twistedmatrix/twisted/_build/results?buildId=3717&view=results
-- 
Adi Roiban
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20210410/a5d5c43d/attachment.htm>