[Twisted-Python] Twisted tests on public CI

Glyph Lefkowitz glyph at twistedmatrix.com
Wed Jul 20 17:18:37 MDT 2016


> On Jul 20, 2016, at 3:18 PM, Adi Roiban <adi at roiban.ro> wrote:
> 
> 
> 
> On 20 July 2016 at 22:36, Glyph Lefkowitz <glyph at twistedmatrix.com <mailto:glyph at twistedmatrix.com>> wrote:
> 
>> On Jul 20, 2016, at 11:59 AM, Adi Roiban <adi at roiban.ro <mailto:adi at roiban.ro>> wrote:
>> 
>> Hi,
>> 
>> We now have Ubuntu / Windows Server and OS X running Twisted tests for each PR, using Travis-CI and Appveyor
>> 
>> Non-`Twisted GitHub Team` members can now run a pretty extensive test suite for commits associated with a PR. 
>> 
>> The coverage report is 90.52% (+/- 0.01%) just by running tests on the public CI.
>> As far as I can tell this is better than what we get with Buildbot builders (since we no longer have BSD slaves and Win7 coverage reporter is broken and Win2012 slave was removed)
> 
> It seems like the only thing that Buildbot is now covering which this setup is not is alternate kernels, which have not diverged in behavior from Twisted's perspective for 7 or 8 years.  I am starting to wonder if we could get rid of the entire current buildbot environment with all its complicated trust issues and just start over with the parts that we actually want: on-demand, latent FreeBSD and Windows builders, for example.  This is a tricky problem, but if it's the *only* thing we're trying to get buildbot to do it's a lot simpler to start there.
> 
> Someone please correct me if I'm wrong though ;).
> 
> 
> +1 to Amber's remark ... different distro have different OpenSSL
>  
> I don't know which OpenSSL versions are supported by latest Twisted.
> 
> For the kernel, I don't think that there are big differences.... so maybe we can have docker based latent slaves to run all these distros and destroy them after each run.
> 
> The docker slaves should keep the Rackspace invoice under control.

This is exactly where I was going, with one important caveat - Travis-CI supports running arbitrary docker images: <https://docs.travis-ci.com/user/docker/> so we do not need buildbot for this :-).

> api-documentation is still executed on Buildbot ... but there is a ticket in the review queue to move it to Travis-CI
> https://twistedmatrix.com/trac/ticket/8518 <https://twistedmatrix.com/trac/ticket/8518>
Cool; that one seems pretty straightforward to move.

>> Some tests on Appveyor are still failing. Help here is much appreciated.
>> 
>> We also got some tests failing on OS X but Amber has already started working to fix them https://twistedmatrix.com/trac/ticket/8639 <https://twistedmatrix.com/trac/ticket/8639>
>> 
>> OS X tests results will not block for now a merge, but you will have to wait for the OS X results. In this way we can see how much longer does a test run takes, now that we also have OS X.
>> 
>> Appveyor was pretty fast in the last days... after the move to Rackspace.
> 
> I'm a little confused - does Appveyor run on Rackspace now, or do we run some component of Appveyor on Twisted's Rackspace account?
> 
> I have received a message from AppVeyor saying that they have moved the open source projects to Rackspace ... and after that the time a build was waiting in the queue is much shorter
> 
> https://www.appveyor.com/blog/2016/07/16/migration-to-rackspace <https://www.appveyor.com/blog/2016/07/16/migration-to-rackspace>

Woot!  Go team Rackspace!  Thanks for that link.

>> Travis-CI was a bit slow on the free queue... not sure how fast is the payed queue :)
>> 
>> Please report any issues here or on Twisted Infrastructure tracker https://github.com/twisted-infra/braid/issues <https://github.com/twisted-infra/braid/issues>
> Thanks *again* for the sustained effort on improving the CI situation, Adi.  It seems like we're not quite there yet, but things are really starting to come together.
> 
> I would consider the Travis-CI / AppVeyor integrations still experimental. 
> 
> I am happy that we are moving to a command line driven CI design as it should make it easier to move between CI systems.

The one thing I wish we could do is extract our build information in some structured format we could actually parse, to make moving between CI executors even easier... but that's probably a few years out at least :).

> This can also help if we want to get latent buildslave as by that time we  get the latent slaves we should be familiar with running tests in a disposable VM/container.

Latent builders would certainly improve our ability to use buildbot considerably.

-glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20160720/b715c7e1/attachment-0002.html>


More information about the Twisted-Python mailing list