[Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

Glyph glyph at twistedmatrix.com
Wed Jun 3 01:05:02 MDT 2020



> On May 28, 2020, at 3:13 PM, Tom Most <twm at freecog.net> wrote:
> 
> On Fri, May 22, 2020, at 10:54 PM, Glyph wrote:
>> So, pidfd's cool, we should totally use it. Also we should use posix_spawn and maybe some other stuff too. But I wonder if there's any heuristic we could use to speed up our current strategy, like ordering the to-reap list by putting things with no open FDs at the front of it? And optimistically assuming that once we've found something to reap, maybe we can stop? And maybe it should run in a cooperator, rather than just blocking the reactor indefinitely?
> 
> Those all sound like reasonable optimizations with little downside to me.

Great!  Anybody feel like filing some tickets? :)

>> Honestly it had not occurred to me that people were managing 20k+ python interpreters at a time with spawnProcess (although, yikes, amazing, you've gotta talk more about this application, and what kind of hardware you're on!)
> 
> Nonono, that was a synthetic stress test. It did spawn Python processes, but they immediately called `exec sleep` to conserve memory.
> 
> The result of the test was that the service would run out of memory _well_ before inefficiencies in Twisted's reaping scheme became a problem.


Aah, that's less exciting ;).

-g


More information about the Twisted-Python mailing list