[Twisted-Python] Using reactor with multiple processes

[Phil Mayers]

Dimitri Vorona wrote:
> Hi,
> 
> i'd like to be able to start additional instances of my server using
> multiprocess module. To do it i just start a new Process with
> mutiprocessing and then create a new instance of my Application there.
> Everything works fine, but i can't figure out a way to
> stop such a process. If I run reactor.stop() in the child process I
> get an infinite loop of errors looking like this:
> 
>    --- <exception caught here> ---
>  File "/usr/share/exe/twisted/internet/tcp.py", line 755, in doRead
>    skt, addr = self.socket.accept()
>  File "/usr/lib/python2.6/socket.py", line 195, in accept
>    sock, addr = self._sock.accept()
> <class 'socket.error'>: [Errno 22] Invalid argument
> 
>  I couldn't not find much information on using reactor from different
> processes, so i'm asking for help here.

I doubt this will work.

When are you starting the multiprocess children?

If it's after the main reactor has started, then you'll run into all 
kinds of problems. When I last looked into it, multiprocess just calls 
"fork()" and then carries on going - it doesn't re-exec the python 
interpreter, so the child process and parent process share state, 
including open file descriptors and such - closing the child process 
will cause "damage" to file descriptors in the parent, amongst other things.

Aside from all that, multiprocess is blocking so doesn't play well with 
Twisted.

You should look into something like Ampoule:

https://launchpad.net/ampoule

...which will work "correctly" with Twisted.