[Twisted-Python] epoll keep sharing state between process even after fork.

[Tobias Oberstein]

Hi,

this is funny, I am also working in this area currently.

Though I use spawnProcess for various reasons, not multiprocessing - which is also the recommendation on the ticket. And it makes sense.

Nevertheless I'd be interested what happens if you try that on a kqueue-reactor OS .. ideally FreeBSD. OSX kqueue isn't the greatest.

/Tobias

Von: twisted-python-bounces at twistedmatrix.com [mailto:twisted-python-bounces at twistedmatrix.com] Im Auftrag von Flint
Gesendet: Mittwoch, 23. Oktober 2013 17:46
An: twisted-python at twistedmatrix.com
Betreff: [Twisted-Python] epoll keep sharing state between process even after fork.

Hi everybody

I a came across a surprising problem when using the epoll based reactor.
(ticket here https://twistedmatrix.com/trac/ticket/6796)

As you can see on the ticket, the epoll object seems to share some state even after fork.
Which means even after having forked the process, making some changes to the inherited epoll object in one process may impact the one existing in another process !

This problem is only related to epoll based reactor, poll and select behave correctly.
I'm aware that some may say that this in not a twisted related problem (but an epoll issue), but I'm addressing my question here to try to figure out what would be the best workaround for this problem in a twisted based project ?

And for information this is what I have already tryed:
- I though about using poll or select reactor, but this is not possible, cause as i'm writing a library I do not decide which reactor will be installed by the person who want to use it.
(Besides as epoll is now the default reactor installed, I certainly want my library to work with it !)

- Using spawnProcess as mentioned in the ticket comment, is not an option too, in my case, I need to share some state between the main process and subprocess. (I have on object in main process space that I want to inherit in all subprocess)

- I (desperately) tryed to re-initialize the reactor._poller object after each fork to set it with a new object, but it was just a very bad idea ! :)

Thank you, in advance, for any possible clue !
(for information, my project is here https://github.com/Grindizer/scaletix)






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20131023/685d2b7c/attachment.html>