[Twisted-Python] operations: baby steps. buildbot move?

Glyph Lefkowitz glyph at twistedmatrix.com
Sun Jan 31 17:46:12 MST 2016


In the spirit of having more of our discussions of what's going on with the operations of twistedmatrix.com <http://twistedmatrix.com/> happening out in the open so that folks can jump in and help, we are going to start discussing this stuff on the public mailing list instead of in private side discussions or the administrative list.  Credentials and other operationally sensitive things will of course still be sent via private GPG-encrypted messages but general "what's going on" stuff should be here.

Right now we have a creaky old machine (dornkirk) running in a datacenter operated by Tummy.  Although they have been generous with discounts, we do have actually free hardware elsewhere, including some sponsored by my employer (Rackspace).  I would like to get everything off of Tummy's hardware for several reasons:

we actually have to pay for tummy's hosting, which is a drain (albeit a minor one) on our very taxed resources from the Conservancy.  (hey slackers: get out your credit cards and head to the sponsor form twistedmatrix.com <http://twistedmatrix.com/> or the site is literally going to go down one day).
by co-locating so many services we can't upgrade the host operating system as often as we'd like without fear of breaking something else
buildbot's TLS cert is expired and I'd like to use Let's Encrypt to get a new one, but the way the web server is configured makes this a bit tricky; it would be easier if it lived on its own host.
the buildmaster is slow and it would be nice if it were on faster hardware
given that many of our builders themselves are running in the cloud, we'd have lower latency if the buildmaster actually ran there

So at some point in the coming week I'd like to just move exactly the version of the Buildmaster we're running right now to a new cloud host on Rackspace; probably the smallest one that comes with an SSD.

Hawkowl is in the process of upgrading us to Buildbot 9 which will be wonderful when it happens, but I want to be clear that this effort is wholly independent from that and neither depends on the other :-).

It looks to me like the thing I need to do is 'fab config.production buildbot.stop; fab config.production buildbot.dump', then make the relevant DNS change.  At this point I think I need to add a new environment to ENVIRONMENTS and do 'fab config.buildbot buildbot.dump'.  It's not obvious to me how to have two hosts present in 'config.production' but if somebody knows this please let me know.

If you're interested in getting involved in operational stuff to help us out with this, I'll be available for pair-opsing (what do you call pair programming when it's ops?) so just get in touch.

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20160131/9e061af9/attachment.html>


More information about the Twisted-Python mailing list