[Twisted-Python] Twisted 16.7.0rc1 Release Candidate Announcement

[Glyph Lefkowitz]

> On Dec 29, 2016, at 4:20 PM, Tim Allen <screwtape at froup.com> wrote:
> 
> On Thu, Dec 29, 2016 at 05:27:44AM -0800, Glyph Lefkowitz wrote:
>> Increasingly, we are assuming a pip-like packaging toolchain for
>> dealing with Twisted's dependencies, so getting familiar with this
>> stuff - pip, requirements.txt, pinning, virtualenv - is worthwhile.
>> (It'll make your life easier in more ways than one.)
> 
> I wasn't going to mention it (because we found a workaround), but since
> the topic has come up...
> 
> My current employer has a fully pip-based packaging toolchain for all
> our internal Python apps, where we build wheels for our apps and all
> their dependencies, and then deploy them into a virtualenv in
> production. For reproducability reasons, we have a PyPI mirror inside
> the corporate firewall, and our wheel-building Docker image has
> a `pip.conf` configured to point at it.
> 
> Twisted 16.6 broke this system when it added a package named
> "incremental" to the `setup_requires` list in setup.py. Because
> `setup_requires` is a feature provided by setuptools, packages listed
> there are installed by (the same machinery as) setuptools' `ez_install`
> command, which does *not* respect `pip.conf`, and so it tried to
> download the package from upstream PyPI and timed out banging its head
> against the firewall.

Thanks, this is very useful information.

If you wanted to have this verified for the future, a network-isolated buildbot would be a good way to ensure we don't mess this up in other ways.  Based on this thread it seems that multiple users have this concern, so it's certainly not specific to you, either.

It would also force us to fix the dang documentation-generator test cases that keep failing when a certain someone forgets to update their SSL cert :-).

> As a workaround, we messed with our build-scripts to manually "pip
> install" incremental before installing Twisted, or anything that depends
> on it, so everything's fine again.
> 
> The Python Packaging Authority already has plans for replacing
> `setup_requires` with something more Pip-friendly (see PEP 518), so over
> time this won't be an issue. For the short term, though, I hope there
> won't be too many more things added to `setup_requires` without
> announcement.

Incremental is a bit of a special case, since "version definition" is one of the few things that needs synchronization between setup-time and run-time.  I was pretty nervous about adding `setup_requires`, but the value of getting our versioning logic outside of Twisted itself seemed worth the potential packaging fallout.

So, while you can expect to see many more things showing up in install_requires unannounced, as we split out low-level portions of Twisted and adopt libraries like h11, I doubt that you'll see another setup_requires arrive, especially before PyPA gets together a better approach than the literal setup.py setup_requires.  Given that you've specifically asked for notice, any project member submitting or reviewing a patch that adds one should give a shout here.

-glyph