[Twisted-Python] Fork/Spawn children to accept connections on the same port.

[Jean-Paul Calderone]

On Fri, 10 Apr 2009 10:16:06 -0700, Eric York <eyork at apple.com> wrote:
>
>>
>>From: Jean-Paul Calderone <exarkun at divmod.com>
>>Date: April 10, 2009 8:57:19 AM PDT
>>To: Twisted general discussion <twisted-python at twistedmatrix.com>
>>Subject: Re: [Twisted-Python] Fork/Spawn children to accept  connections on 
>>the same port.
>>Reply-To: Twisted general discussion <twisted- python at twistedmatrix.com>
>>
>>
>>On Mon, 6 Apr 2009 15:25:39 -0700, Eric York <eyork at apple.com> wrote:
>>>I am trying to get the highest level of performance using all of  the 
>>>processors cores on a server.
>>>
>>>In the past, a unix app would bind/listen to a socket and then fork  or 
>>>spawn children to accept connections on that socket. I can’t see  how  to 
>>>do that in Twisted. Can someone point me in the right  direction?
>>>
>>>The solution of using a single process to accept connections and  then 
>>>farm out work to child processes, while a workable solution,  isn’t at 
>>>the same level of performance as children processes that  are doing  their 
>>>own accepts.
>
>>It's also possible to bind a port in one process and then send it
>>over a UNIX socket to another process; this comes closer to the bind/ fork
>>approach, but accomplishes the resource sharing explicitly via fd  passing
>>rather than through fork.
>
>
>This is the path that I would like to follow. I can see how to spawn a 
>child process and pass the fd in Twisted. What I can't see how to do  in 
>Twisted is to have a parent process just bind and listen to a  socket and 
>have a child process accept on that socket. In twisted/ internet/tcp.py in 
>the class Port, there is a function startListening  which does 
>bind/listen/startReading all in this one function. It seems  that a small 
>refactoring would allow the parent to bind and a child to  do startReading, 
>if the call to startReading was moved out of  startListening. The reactor 
>calls would also need a small refactoring  to allow this type of setup. How 
>does that sound?

The main problem that comes up with this approach is that little, or possibly
none, of Twisted is written to be fork-safe.  As with most programs, though,
this is /usually/ not a problem.  However, there are some areas where it can
be.  For example, if you use epoll reactor, the internal epoll file descriptor
may end up shared between the two processes.  This results in weird behavior
like thread wakeup messages in one process waking up the reactor thread in
the other process.  It's probably possible to fix all the problems like this,
but rather than try, I'd just not try forking a Twisted process without using
exec in one of the resulting processes shortly afterwards.

If you create a port in one process, you can just grab its fileno and pass
that to another process.  Then you can create a new Port object with that
fileno.  This may involve subclassing Port and overriding
createInternetSocket to just return the fileno you have already.  These APIs
aren't really intended for direct use by applications, but there's not really
another way to do what you want right now.  You may want to help us work out
a better API for creating Ports and such from pre-existing sockets if you go
this route so that there's less chance of some deep-down implementation
change disturbing your app.

Jean-Paul