Add tox environments for all buildbot builders

[Adi Roiban]

windows7-64-py2.7-coverage and ubuntu16.04-py2.7-nomodules-coverage builder are running without tox

we need to create tox environment for them so that we can update buildbot to execute them using tox

in this way we should have a more consistent builbot configuration

and if we are doing this, we should check that all builders have a tox environment... with the exception of the benchmark builders

here is the list of buildbot builders not running using tox:

• ubuntu16.04-py2.7-nomodules-coverage
• windows7-64-py2.7-coverage
• windows7-64-py2.7-msi
• windows7-64-py2.7
• documentation

[Adi Roiban]

PR at ​https://github.com/twisted/twisted/pull/280

this will break on buildbot as it needs an update from to tox side

if everything is ok I will consider updating tox.ini so that it will export all environments

in this this branch can be merged using the old environments and then we update buildbot and can create a new ticket to remove the old environments

[Adi Roiban]

Please check that all is ok. Thanks!

As commented before, if you are happy with this new code I will try to re-write in a way in which the old environments are also available

[Craig Rodrigues]

I managed to find an old Windows laptop, and installed Windows 10 on it, then installed Python 3.5.

I managed to pip install tox. However, when I ran tox, I got this error:

OSError: [WinError 193] %1 is not a valid Win32 application

Did you encounter that? It seems to be a known problem. According to this link: ​https://bitbucket.org/hpk42/tox/issues/257/shell-commands-fail-on-windows-due-to

I looked for this code in tox:

 return self.session.popen(args, shell=False, cwd=str(cwd),
                                             universal_newlines=True,
                                             stdout=stdout, stderr=stderr, env=env)

and replaced it with:

 return self.session.popen(args, shell=(sys.platform == 'win32'), cwd=str(cwd),
                                             universal_newlines=True,
                                             stdout=stdout, stderr=stderr, env=env)

and tox seemed to run after that.

[Craig Rodrigues]

OK, so I read more of the audit trail of ​​https://bitbucket.org/hpk42/tox/issues/257/shell-commands-fail-on-windows-due-to

The tox developer didn't accept that hack to tox.

Basically the issue is this. On Windows, if you have a script FOOBAR.CMD, you need to specify the full name for it in the tox.ini script, i.e. FOOBAR.CMD, and not FOOBAR.

I definitely ran into this problem when trying to run the Twisted tests under Windows with tox. I think the tox.ini file in Twisted needs to be modified to invoke the full name of the trial command in Windows.

Once 8491 is implemented and trial becomes a console_script, this will be less of a problem.

[Craig Rodrigues]

This needs to be revisited now that extras works on Python 3 7807

[Craig Rodrigues]

Also, do you have test results from running this under tox on Windows? See my previous comment

[Adi Roiban]

Thanks Craig for the review.

I have pushed more changes... this time with backward compatibility for the existing buildbot configuration.

I hope that the new tox definition makes more sense.

Here are the new tox environments for the existing buildbot:

• ubuntu16.04-py2.7-nomodules-coverage -> py27-nodeps-withcov-posix
• windows7-64-py2.7-coverage -> py27-alldeps-withcov-windows
• windows7-64-py2.7-msi -> py27-wheel
• windows7-64-py2.7 -> py27-alldeps-nocov-windows
• documentation -> apidocs,narrativedocs (already defined, without publish)

For a better coverage we should also have a py27-nodeps-withcov-windows builder on appveyor, beside the bdist which is targeted by #8676

I have created unsupported builders using the new environments. You can see them here

​https://buildbot.twistedmatrix.com/boxes-unsupported?branch=8544-tox-for-buildbot

[Adi Roiban]

OK.

The tox/venv builder are working... now, the fail on Windows due to virtualenv --clear issue, but there is nothing we can do on the twisted side to fix it.

the fix should be done on the buildbot config... and buildbot config depende on this branch.

so I would like to have this branch merged so that we can focus on buildbot or appveyor changes

Thanks!

[hawkowl]

I bumped the buildbot config to fix the --clear issue. I'm happy with this, will merge and update the buildbots to reflect these new builders.

[Amber Brown <hawkowl@…>]

In b8a241c:

Error: Processor CommitTicketReference failed

 does not appear to be a Git repository. See the log for more information.