[Twisted-Python] Deadlocks when launching processes - how to investigate?

[exarkun at twistedmatrix.com]

On 02:50 am, glyph at twistedmatrix.com wrote:
>
>On Feb 18, 2014, at 6:07 PM, exarkun at twistedmatrix.com wrote:
>>On 18 Feb, 07:29 pm, glyph at twistedmatrix.com wrote:
>>>
>>>On Feb 15, 2014, at 5:32 PM, exarkun at twistedmatrix.com wrote:
>>>>On 15 Feb, 07:58 pm, glyph at twistedmatrix.com wrote:
>>>>>
>>>>>The one thing that confused me was that the sample program appeared 
>>>>>to be running the program only once a second, and waiting for it to 
>>>>>complete before running it again.
>>>>
>>>>I think it's more like 81 processes once a second and *not* waiting 
>>>>for them to complete before starting over.  Notice the lack of 
>>>>yields in key places.  I suspect inlineCallbacks has gradually eaten 
>>>>out the part of your brain that recognizes that keyword. ;)
>>>
>>>I sort of noted this on the ticket, but I think the idea of using 
>>>KQueue to address this would be great.  Is there a similar thing we 
>>>might be able to do on Linux to get rid of the dependence on a 
>>>SIGCHLD handler?
>>
>>Not that I know of.  As far as I know Linux is missing good child 
>>process event notification support.
>
>Wait... what about... signalfd?!
>
>Lots of mentions of SIGCHLD here:
>
><http://linux.die.net/man/2/signalfd>

It's sort of problematic for a library.  You have to block the signal 
for signalfd to work reliably.  Now no other code that depends on 
SIGCHLD can run properly in that process.

We could simply declare that to be the case and go ahead... But based on 
past experience I'm sure some people would be unhappy with that.

Perhaps it could be an option only turned on by applications that know 
it's okay for them and that want to avoid the mess of signals?

On the other hand we seem to be dealing with the mess of signals on 
Linux pretty okay at the moment.  Who would opt in to this and why?

Jean-Paul