[Twisted-Python] epoll keep sharing state between process even after fork.

[Phil Mayers]

On 23/10/13 16:46, Flint wrote:
> Hi everybody
>
> I a came across a surprising problem when using the epoll based reactor.
> (ticket here https://twistedmatrix.com/trac/ticket/6796)
>
> As you can see on the ticket, the epoll object seems to share some state
> even after fork.
> Which means even after having forked the process, making some changes to
> the inherited epoll object in one process may impact the one existing in
> another process !
>
> This problem is only related to epoll based reactor, poll and select
> behave correctly.
> I'm aware that some may say that this in not a twisted related problem
> (but an epoll issue), but I'm addressing my question here to try to
> figure out what would be the best workaround for this problem in a
> twisted based project ?

The problem is that you're sharing the epoll object (and thus the 
underlying epoll FD, and associated state) with the multiprocessing 
child processes.

Either:

  1. Don't use multiprocessing
  2. Arrange for the epoll object (or FD) to be closed after fork, but 
before exec, so that the child process can't fiddle with it
  3. Create the epoll object after starting the multiprocessing children

IMHO Twisted and multiprocessing are not good together.