Opened 4 years ago

Last modified 22 months ago

#7945 enhancement new

— at Twisted's external C modules might be worth putting in an external packageVersion 4

Reported by: hawkowl Owned by:
Priority: normal Milestone:
Component: core Keywords:
Cc: Branch: branches/external-cexts-7945
branch-diff, diff-cov, branch-cov, buildbot
Author: hawkowl

Description (last modified by hawkowl)

This would involve:

  • Creating a new package, with the C-exts
  • Removing them from Twisted
  • Import them into Twisted from the external package

As to why:

Twisted has a handful of (mostly optional) C extension modules, to provide support for the IOCP reactor, some inetd stuff, sendmsg on Py2, and initgroups on Py2.6, among others. The vast majority of Twisted's users do not require these C modules, and it only makes it harder to install (requiring python-dev + a compiler). If they were bundled into another package and distributed on PyPI, then it would make the majority case simpler, but still allowing easy access to the extension modules.

This has the following direct benefits:

  • Twisted users (Tahoe, Crossbar/Autobahn, Buildbot) no longer need to have a C compiler to install Twisted. (It will still be required for things like PyOpenSSL, but that is an optional dependency)
  • Twisted can distribute a single wheel and a single source distribution on PyPI. The msi and exe installers and the Windows wheel will only be the C extensions, and therefore much smaller. Since these rarely change, they will not need to be changed on pypi for a long while.
  • We can add extra C modules to this package, without having to hard-depend on CFFI in the future.

Change History (4)

comment:1 Changed 4 years ago by hawkowl

Author: hawkowl
Branch: branches/external-cexts-7945

(In [45090]) Branching to external-cexts-7945.

comment:2 Changed 4 years ago by hawkowl

Keywords: review added

This is a branch that does this. It removes the support code from Twisted, and puts it in a new package.

This will not work on the buildbots without some build script modification. The way to test it manually is (from a Twisted checkout):

virtualenv venv
venv/bin/pip install zope.interface
cd cexts
../venv/bin/python setup.py install
cd ..
venv/bin/python bin/trial twisted

To test something that specifically has cexts:

venv/bin/python bin/trial twisted.python.test.test_sendmsg twisted.python.test.test_util.InitGroupsTests

This branch may be backwards incompatible.

comment:3 Changed 4 years ago by hawkowl

This is related to #3586.

comment:4 Changed 4 years ago by hawkowl

Description: modified (diff)
Note: See TracTickets for help on using tickets.