Opened 6 years ago

Closed 6 years ago

#8544 enhancement closed fixed (fixed)

Add tox environments for all buildbot builders

Reported by: Adi Roiban Owned by: hawkowl
Priority: normal Milestone:
Component: core Keywords:
Cc: Branch: 8544-tox-for-buildbot
branch-diff, diff-cov, branch-cov, buildbot
Author:

Description (last modified by Adi Roiban)

windows7-64-py2.7-coverage and ubuntu16.04-py2.7-nomodules-coverage builder are running without tox

we need to create tox environment for them so that we can update buildbot to execute them using tox

in this way we should have a more consistent builbot configuration

and if we are doing this, we should check that all builders have a tox environment... with the exception of the benchmark builders

here is the list of buildbot builders not running using tox:

  • ubuntu16.04-py2.7-nomodules-coverage
  • windows7-64-py2.7-coverage
  • windows7-64-py2.7-msi
  • windows7-64-py2.7
  • documentation

Change History (10)

comment:1 Changed 6 years ago by Adi Roiban

Branch: 8544-tox-for-buildbot
Description: modified (diff)
Owner: set to Adi Roiban

PR at https://github.com/twisted/twisted/pull/280

this will break on buildbot as it needs an update from to tox side

if everything is ok I will consider updating tox.ini so that it will export all environments

in this this branch can be merged using the old environments and then we update buildbot and can create a new ticket to remove the old environments

comment:2 Changed 6 years ago by Adi Roiban

Keywords: review added
Owner: Adi Roiban deleted
Summary: Add tox configuration for windows7-64-py2.7-coverage and ubuntu16.04-py2.7-nomodules-coverageAdd tox environments for all buildbot builders

Please check that all is ok. Thanks!

As commented before, if you are happy with this new code I will try to re-write in a way in which the old environments are also available

comment:3 Changed 6 years ago by Craig Rodrigues

I managed to find an old Windows laptop, and installed Windows 10 on it, then installed Python 3.5.

I managed to pip install tox. However, when I ran tox, I got this error:

OSError: [WinError 193] %1 is not a valid Win32 application

Did you encounter that? It seems to be a known problem. According to this link: https://bitbucket.org/hpk42/tox/issues/257/shell-commands-fail-on-windows-due-to

I looked for this code in tox:

 return self.session.popen(args, shell=False, cwd=str(cwd),
                                             universal_newlines=True,
                                             stdout=stdout, stderr=stderr, env=env)

and replaced it with:

 return self.session.popen(args, shell=(sys.platform == 'win32'), cwd=str(cwd),
                                             universal_newlines=True,
                                             stdout=stdout, stderr=stderr, env=env)

and tox seemed to run after that.

comment:4 Changed 6 years ago by Craig Rodrigues

OK, so I read more of the audit trail of ​https://bitbucket.org/hpk42/tox/issues/257/shell-commands-fail-on-windows-due-to

The tox developer didn't accept that hack to tox.

Basically the issue is this. On Windows, if you have a script FOOBAR.CMD, you need to specify the full name for it in the tox.ini script, i.e. FOOBAR.CMD, and not FOOBAR.

I definitely ran into this problem when trying to run the Twisted tests under Windows with tox. I think the tox.ini file in Twisted needs to be modified to invoke the full name of the trial command in Windows.

Once 8491 is implemented and trial becomes a console_script, this will be less of a problem.

comment:5 Changed 6 years ago by Craig Rodrigues

Owner: set to Adi Roiban

This needs to be revisited now that extras works on Python 3 7807

comment:6 Changed 6 years ago by Craig Rodrigues

Keywords: review removed

Also, do you have test results from running this under tox on Windows? See my previous comment

Last edited 6 years ago by Craig Rodrigues (previous) (diff)

comment:7 Changed 6 years ago by Adi Roiban

Description: modified (diff)

Thanks Craig for the review.

I have pushed more changes... this time with backward compatibility for the existing buildbot configuration.

I hope that the new tox definition makes more sense.

Here are the new tox environments for the existing buildbot:

  • ubuntu16.04-py2.7-nomodules-coverage -> py27-nodeps-withcov-posix
  • windows7-64-py2.7-coverage -> py27-alldeps-withcov-windows
  • windows7-64-py2.7-msi -> py27-wheel
  • windows7-64-py2.7 -> py27-alldeps-nocov-windows
  • documentation -> apidocs,narrativedocs (already defined, without publish)

For a better coverage we should also have a py27-nodeps-withcov-windows builder on appveyor, beside the bdist which is targeted by #8676

I have created unsupported builders using the new environments. You can see them here

https://buildbot.twistedmatrix.com/boxes-unsupported?branch=8544-tox-for-buildbot

Last edited 6 years ago by Adi Roiban (previous) (diff)

comment:8 Changed 6 years ago by Adi Roiban

Keywords: review added
Owner: Adi Roiban deleted

OK.

The tox/venv builder are working... now, the fail on Windows due to virtualenv --clear issue, but there is nothing we can do on the twisted side to fix it.

the fix should be done on the buildbot config... and buildbot config depende on this branch.

so I would like to have this branch merged so that we can focus on buildbot or appveyor changes

Thanks!

comment:9 Changed 6 years ago by hawkowl

Keywords: review removed
Owner: set to hawkowl

I bumped the buildbot config to fix the --clear issue. I'm happy with this, will merge and update the buildbots to reflect these new builders.

comment:10 Changed 6 years ago by Amber Brown <hawkowl@…>

Resolution: fixed
Status: newclosed

In b8a241c:

Error: Processor CommitTicketReference failed
 does not appear to be a Git repository. See the log for more information.
Note: See TracTickets for help on using tickets.