[Twisted-Python] Twisted in a multicore environment

[Laurens Van Houtven]

FWIW,  I have used Ampoule to great effect, but as JP points out it's hardly the only option. You're bound to end up with some measure of multiprocessing. Bear in mind that not all workloads are well-suited for this kind of problem! Always measure before deciding to make your codebase that much more complicated :)


cheers
lvh



On 10 Jul 2012, at 18:03, exarkun at twistedmatrix.com wrote:

> On 03:14 pm, augustocaringi at gmail.com wrote:
>> Hi,
>> 
>>   I'm researching the best way to implement/use a Twisted-based
>> server in a multicore environment...
>> 
>>   There is the Ampoule project, that I realize is considered the
>> best way to do that. Right?
> 
> It's a way.  "Best" depends on the details and goals of the project.
> 
> Here's a stackoverflow question/answer on basically the same topic.  In 
> particular, it specifically answers the question of a listening port 
> shared between multiple processes and gives examples of how to do this:
> 
>  http://bit.ly/MiCHtQ
> 
> Jean-Paul
>>   I'm also reading about the internals of Nginx HTTP server. This
>> server utilizes the same reactor pattern of Twisted (epoll based)...
>> 
>>   "What resulted is a modular, event-driven, asynchronous,
>> single-threaded, non-blocking architecture which became the foundation
>> of nginx code." http://www.aosabook.org/en/nginx.html
>> 
>>   But to maximize the use of processors in a multicore environment,
>> Nginx do this:
>> 
>>   "nginx doesn't spawn a process or thread for every connection.
>> Instead, worker processes accept new requests from a shared "listen"
>> socket and execute a highly efficient run-loop inside each worker to
>> process thousands of connections per worker"
>> 
>>   My question: There is something similar in Twisted? Or do you
>> think that is easy to implement something like that?
>> 
>>   Thanks!
>> 
>> --
>> Augusto Mecking Caringi
>> 
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python