[Twisted-Python] Using reactor with multiple processes

[David Yoakley]

Dimitri Vorona wrote:
> Hi,
>
> i'd like to be able to start additional instances of my server using
> multiprocess module. To do it i just start a new Process with
> mutiprocessing and then create a new instance of my Application there.
> Everything works fine, but i can't figure out a way to
> stop such a process. If I run reactor.stop() in the child process I
> get an infinite loop of errors looking like this:
>

This is a very timely discussion for us.  We are trying to launch multiple
independent twistedized children with python multiprocess and yes (as Phil
noted) we are getting very strange results.  We were aware the multiprocess
module does a fork to spawn children but we hoping that this would not be an
issue for us because our parent process does NOT start a reactor before
spawning the children.  In fact even if our parent process never starts its
reactor, as soon as we spawn the 2nd twistedized child, bad things start
happening in the children.   The parent does import the child modules in
order to start them and since each child module also imports twisted, we are
thinking that some twisted state gets established just on the import of the
twisted reactor into the parent and then the fork clones that state into the
children.

Thanks Phil for the reference to ampoule.   We will look at that next.  We
are still hoping to get the parent set up in such a way that whatever the
forking :-) shared state is, it does not get established until after the
children are spawed. I'll post back later with what we find.  Phil your
thoughts on this would also be appreciated.

Cheers,
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20090916/819d13b5/attachment.htm