[Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

Glyph glyph at twistedmatrix.com
Fri May 22 23:54:42 MDT 2020



> On May 19, 2020, at 11:14 AM, Tom Most <twm at freecog.net> wrote:
> 
> On Tue, May 19, 2020, at 5:41 AM, Jean-Paul Calderone wrote:
>> On Tue, May 19, 2020 at 8:33 AM Barry Scott <barry.scott at forcepoint.com <mailto:barry.scott at forcepoint.com>> wrote:
>> Why not waitpid for all children (pid=0 or pid=-1) in a single call? 
>> 
>> Because then you might reap a child process someone launched with a different API - eg the Python subprocess module.
> 
> Exactly. I am currently using this as a workaround, but it's not a general solution. You need to be careful because subprocess use hides in many places. For example, beware uuid.uuid1().

So, pidfd's cool, we should totally use it.  Also we should use posix_spawn and maybe some other stuff too.  But I wonder if there's any heuristic we could use to speed up our current strategy, like ordering the to-reap list by putting things with no open FDs at the front of it?  And optimistically assuming that once we've found something to reap, maybe we can stop?  And maybe it should run in a cooperator, rather than just blocking the reactor indefinitely?  Honestly it had not occurred to me that people were managing 20k+ python interpreters at a time with spawnProcess (although, yikes, amazing, you've gotta talk more about this application, and what kind of hardware you're on!)

Anyway there's lot of exciting room for performance improvements here, it would be nice to have some benchmarks for this kind of high-load process-monitor application.

-g
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20200522/1b75ce75/attachment.htm>


More information about the Twisted-Python mailing list