[Twisted-Python] Twisted in a multicore environment
dynamicgl at gmail.com
Sat Jul 14 06:39:32 EDT 2012
On Wed, Jul 11, 2012 at 2:08 AM, Werner Thie <werner at thieprojects.ch> wrote:
> On 7/10/12 6:17 AM, Laurens Van Houtven wrote:
> > FWIW, I have used Ampoule to great effect, but as JP points out it's
> hardly the only option. You're bound to end up with some measure of
> multiprocessing. Bear in mind that not all workloads are well-suited for
> this kind of problem! Always measure before deciding to make your codebase
> that much more complicated :)
> > cheers
> > lvh
> > On 10 Jul 2012, at 18:03, exarkun at twistedmatrix.com wrote:
> >> On 03:14 pm, augustocaringi at gmail.com wrote:
> >>> Hi,
> >>> I'm researching the best way to implement/use a Twisted-based
> >>> server in a multicore environment...
> >>> There is the Ampoule project, that I realize is considered the
> >>> best way to do that. Right?
> >> It's a way. "Best" depends on the details and goals of the project.
> >> Here's a stackoverflow question/answer on basically the same topic. In
> >> particular, it specifically answers the question of a listening port
> >> shared between multiple processes and gives examples of how to do this:
> >> http://bit.ly/MiCHtQ
> >> Jean-Paul
> >>> I'm also reading about the internals of Nginx HTTP server. This
> >>> server utilizes the same reactor pattern of Twisted (epoll based)...
> >>> "What resulted is a modular, event-driven, asynchronous,
> >>> single-threaded, non-blocking architecture which became the foundation
> >>> of nginx code." http://www.aosabook.org/en/nginx.html
> >>> But to maximize the use of processors in a multicore environment,
> >>> Nginx do this:
> >>> "nginx doesn't spawn a process or thread for every connection.
> >>> Instead, worker processes accept new requests from a shared "listen"
> >>> socket and execute a highly efficient run-loop inside each worker to
> >>> process thousands of connections per worker"
> >>> My question: There is something similar in Twisted? Or do you
> >>> think that is easy to implement something like that?
> >>> Thanks!
> >>> --
> >>> Augusto Mecking Caringi
> We observed really great scaling on multi cores with moving the
> application part either to ampoule for PDF production or in the other
> case I wrote an implementation of self regulating process pool based on
> spread, leaving only the serving to twisted in both cases.
> With handing work out to other processes you get another benefit which
> is isolation of python, which is the only way to use a package, like
> reportlab which survives no sort of reentrancy, for a webservice.
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
I want to know whether you have experienced any serious bug with
ampoule? What version of ampoule did you use? Is it 0.2.0? Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Twisted-Python