[Twisted-Python] Fork/Spawn children to accept connections on the same port.

[Eric York]

>
> From: Jean-Paul Calderone <exarkun at divmod.com>
> Date: April 10, 2009 8:57:19 AM PDT
> To: Twisted general discussion <twisted-python at twistedmatrix.com>
> Subject: Re: [Twisted-Python] Fork/Spawn children to accept  
> connections on the same port.
> Reply-To: Twisted general discussion <twisted- 
> python at twistedmatrix.com>
>
>
> On Mon, 6 Apr 2009 15:25:39 -0700, Eric York <eyork at apple.com> wrote:
>> I am trying to get the highest level of performance using all of  
>> the processors cores on a server.
>>
>> In the past, a unix app would bind/listen to a socket and then fork  
>> or spawn children to accept connections on that socket. I can’t see  
>> how  to do that in Twisted. Can someone point me in the right  
>> direction?
>>
>> The solution of using a single process to accept connections and  
>> then  farm out work to child processes, while a workable solution,  
>> isn’t at  the same level of performance as children processes that  
>> are doing  their own accepts.

> It's also possible to bind a port in one process and then send it
> over a UNIX socket to another process; this comes closer to the bind/ 
> fork
> approach, but accomplishes the resource sharing explicitly via fd  
> passing
> rather than through fork.


This is the path that I would like to follow. I can see how to spawn a  
child process and pass the fd in Twisted. What I can't see how to do  
in Twisted is to have a parent process just bind and listen to a  
socket and have a child process accept on that socket. In twisted/ 
internet/tcp.py in the class Port, there is a function startListening  
which does bind/listen/startReading all in this one function. It seems  
that a small refactoring would allow the parent to bind and a child to  
do startReading, if the call to startReading was moved out of  
startListening. The reactor calls would also need a small refactoring  
to allow this type of setup. How does that sound?

Eric

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20090410/39d4dbe4/attachment.html>