ContinuousIntegration/TestSlaveConfiguration

Version 22 (modified by tom.prince, 20 months ago)

Update information on how to provide a slave

A build slave which is to run the Twisted test suite on a particular platform or in a particular configuration needs the following software:

  •  Python
    • On Debian, the python-profiler package should also be installed.
    • On platforms where the Python headers are separated from the main Python package, these should also be installed (usually python-dev or python-devel).
  • A C compiler which can build Python extension modules (On POSIX, this usually means gcc; on Windows, some version of Visual Studio, depending on which version of Python is installed; most likely  VS2008 aka VS9)
  •  zope.interface (Debian package python-zope.interface)
  • On Windows,  pywin32
  •  Twisted - required by buildbot
  •  BuildBot - only buildbot-slave is necessary
  •  bazaar - sometimes packaged as bazaar-ng

As well as all (or as many as possible) of the libraries Twisted depends on:

  •  PyCrypto (Debian package python-crypto)
  •  pyOpenSSL (depends on  OpenSSL) (Debian package python-pyopenssl)
  •  pyasn1 (Debian package python-pyasn1)
  •  gmpy (depends on  GMP) (Debian package python-gmpy)
  • SOAPpy (Debian package python-soappy)
  • pygtk (depends on gtk2) (Debian package python-gtk2)
  •  subunit (Debian package python-subunit)

Unfortunately a few tests in Twisted's test suite will fail if certain guarantees are not made by the environment. For example, the multicast tests try to use the actual network stack to send multicast packets. Here are some things that are known about this:

  • For the multicast tests to pass, multicast packets must be routable over localhost.
    • On OS X, this means "Internet Sharing" must be disabled
    • Some Linux distributions have a default iptables rule to drop multicast traffic, or require an explicit iptables rule to allow it. For example, on Red Hat:
      -A INPUT --dest 224.0.0.0/4 -j ACCEPT
      

A build slave is created with the buildbot create-slave command. This takes the master address, slave name, and slave password as arguments. The master address is buildbot.twistedmatrix.com:9987. Several other optional arguments are allowed. It is a good idea to specify a value of around 10 for --log-count to avoid unbounded log file growth. If there are special networking considerations, it may also be useful to specify a value for --keepalive. Other options generally aren't needed.

If bzr is being used then on the very first build by a new slave a lengthy bzr initialization step is required. This usually needs to be completed manually after buildbot times out the operation for taking too long. The command which will fail on buildbot is:

bzr checkout svn://svn.twistedmatrix.com/svn/Twisted/trunk Twisted

with a working directory of /path/to/slave-root/builder-name/. The correct directory should have Twisted and .bzr directories in it already. Re-running the failed command in that directory will resume the checkout and eventually succeed (perhaps after quite a long time - dozens of minutes, at least - this is expected).

If you wish to provide a buildslave for twisted, send the follwoing information to buildbot at twisted matrix dot com:

1. Desired slave name.

This will show up on  http://buildbot.twistedmatrix.com/buildslaves Something like <name>-<machine-description>

2. Password

This is sent cleartext.

3. Email address

This is used to send mail messages when the slave disappears. It isn't visible.

4. What builders you want set up.

  • what reactors are available/should be tested
  • path to the python executable, if it isn't the default python on the path.

5. How many parallel builds you want allowed to run.