[Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Barry Scott]

On Monday, 18 May 2020 22:21:16 BST Tom Most wrote:
> On Sun, May 17, 2020, at 12:22 PM, Glyph wrote:
> >> In the one case that may be complicated I think that pinning Twisted will
> >> be acceptable, since the service in question is a process supervisor
> >> without direct external exposure.> 
> > Well in that case I'll just do my best to add lots of goodies (posix_spawn
> > support!) that will make you want to upgrade that to current twisted and
> > python 3.8 :).
> Oh it's not for lack of wanting. :-)
> 
> The service in question is actually spawning Python processes, so
> posix_spawn support doesn't seem likely to help much: fork() + exec() is a
> drop in the bucket compared to Python imports, particularly in Python 3
> (I'm hoping to explore oxidized-importer [1] to help with this at some
> point).

Would putting all the .pyc/.pyo into a zip file help with the import problem?
You only have one file to open, the .zip, and no compile as its only 
.pyc/.pyo.


> 
> If looking improve process management in Twisted, I think that the
> lowest-hanging fruit is to watch children with pidfd, rather than reaping
> all children on SIGCHLD. This O(n) reaping --- one waitpid() call per
> running process --- starts to consume substantial CPU. IIRC it pegged a
> core at 20,000 processes or so.

Why not waitpid for all children (pid=0 or pid=-1) in a single call?

Barry
> 
> ---Tom
> 
> [1]: https://pypi.org/project/oxidized-importer/