Opened 5 years ago

Last modified 3 days ago

#3696 enhancement new

Add extras_require to top-level to list Twisted optional dependancies

Reported by: thijs Owned by:
Priority: normal Milestone:
Component: core Keywords: review
Cc: zooko Branch:
Author: Launchpad Bug:


Setuptools provides a feature for listing optional dependancies: extras_require. It seems like this functionality should be in trunk/twisted/python/ or trunk/twisted/python/ or something, especially given that trunk/twisted/conch/topfiles/ needs to know about the pycrypto dependency somehow too.

Attachments (1)

extra_requires-3696.diff (2.0 KB) - added by herrwolfe 4 days ago.,, news file

Download all attachments as: .zip

Change History (15)

comment:1 Changed 5 years ago by glyph

  • Owner changed from glyph to radix

For previous discussion, see #976, which spawned this ticket.

comment:2 Changed 5 years ago by glyph

Oops I meant #967.

comment:3 Changed 5 years ago by zooko

See also #3238 (patch: declare that twisted requires pywin32 if it is to offer process management or iocp reactor on Windows).

comment:4 Changed 5 years ago by zooko

See also (pycrypto package is required for manhole). We're going to declare that Tahoe-LAFS depends on pycrypto. But that isn't really right. Really it is that Tahoe-LAFS depends on Twisted's "manhole" feature, which depends on pycrypto. So for example, if a future release of Twisted implemented manhole without using pycrypto, then Tahoe-LAFS would not require pycrypto anymore. So the "best" way to solve this is for Tahoe-LAFS to specify that it depends on Twisted plus the "extra" of manhole, and for Twisted to declare that if you want the "extra" of manhole then you need pycrypto.

comment:5 Changed 5 years ago by zooko

  • Cc zooko added

comment:6 Changed 4 years ago by davidsarah

Also, twisted.conch depends on pycrypto and pyasn1.

comment:7 Changed 4 years ago by zooko

Opened #4438 which emphasizes that if Twisted fails to declare an existing dependency then it is a bug, but if Twisted fails to declare a new dependency then it is a regression.

comment:8 Changed 3 years ago by <automation>

  • Owner radix deleted

comment:9 Changed 4 months ago by glyph

#7092 is a duplicate of this.

comment:10 Changed 3 months ago by glyph

#6767 is related to this.

Changed 4 days ago by herrwolfe,, news file

comment:11 Changed 4 days ago by herrwolfe

  • Keywords review added

The patch extra_requires-3696.diff allows the user to install optional dependencies. Since this method only installs the optional dependencies if requested, it seems as though it would make sense if the list were relatively exhaustive. The optional packages I have listed in the patch were specified by exarkun in comment:9:ticket:6767.

This also closes #6767. Using extra_requires seems to be a more manageable solution than project level dev-requirements.txt files. This is because each set of dependencies can be specified in a single top level file. If needed, each project's optional dependencies could be set up as an entry in the extra_requires dictionary.

Here is an example of how this new functionality can be used: in order to install all of the twisted developer tools and tools that help when working with Conch, one would type

pip install -e . "twisted[conch]"
pip install -e . "twisted[dev]"

I am not sure how this should be tested. I would think that buildbots could install package sets and then the current tests could be run as a check. But, I don't know what additional tests could be written.

Also - if this patch is accepted, I think adding a small section to the narrative docs detailing how this new functionality should be used would be helpful.


Last edited 3 days ago by herrwolfe (previous) (diff)

comment:12 Changed 3 days ago by glyph

Thank you very much for contributing this patch, herrwolfe. We really need help in this area; I'll do my best to make sure that this patch gets reviewed soon.

comment:13 Changed 3 days ago by dstufft

Quick thoughts:

Looking at the INSTALL file, pywin32 is not an optional dependency, it is a mandatory dependency but only on Windows. Thus you should do

import sys;

if sys.platform == "win32":
    requirements += ["pywin32"]

Then inside of a setup.cfg you should do

requires-dist =
    zope.interface >= 3.6.0
    pywin32; sys_platform == 'pywin32'

The setup.cfg is for wheel files and it will override the install_requires. It's a duplication but it makes this work better for people.

You also probably want a "ssl" extra that installs pyOpenSSL too.

Note: I've just talked to glyph and i'm not sure if pywin32 is an actual dependency or if INSTALL is just full of lies. I'm leaving this comment here just incase y'all want to do that since it says how to do it.

comment:14 Changed 3 days ago by zooko

According to pywin32 is required if you use (or possibly just import?) endpoints.

Note: See TracTickets for help on using tickets.