[Twisted-Python] Twisted in a multicore environment

[gelin yan]

On Wed, Jul 11, 2012 at 2:08 AM, Werner Thie <werner at thieprojects.ch> wrote:

> On 7/10/12 6:17 AM, Laurens Van Houtven wrote:
> > FWIW,  I have used Ampoule to great effect, but as JP points out it's
> hardly the only option. You're bound to end up with some measure of
> multiprocessing. Bear in mind that not all workloads are well-suited for
> this kind of problem! Always measure before deciding to make your codebase
> that much more complicated :)
> >
> >
> > cheers
> > lvh
> >
> >
> >
> > On 10 Jul 2012, at 18:03, exarkun at twistedmatrix.com wrote:
> >
> >> On 03:14 pm, augustocaringi at gmail.com wrote:
> >>> Hi,
> >>>
> >>>    I'm researching the best way to implement/use a Twisted-based
> >>> server in a multicore environment...
> >>>
> >>>    There is the Ampoule project, that I realize is considered the
> >>> best way to do that. Right?
> >>
> >> It's a way.  "Best" depends on the details and goals of the project.
> >>
> >> Here's a stackoverflow question/answer on basically the same topic.  In
> >> particular, it specifically answers the question of a listening port
> >> shared between multiple processes and gives examples of how to do this:
> >>
> >>   http://bit.ly/MiCHtQ
> >>
> >> Jean-Paul
> >>>    I'm also reading about the internals of Nginx HTTP server. This
> >>> server utilizes the same reactor pattern of Twisted (epoll based)...
> >>>
> >>>    "What resulted is a modular, event-driven, asynchronous,
> >>> single-threaded, non-blocking architecture which became the foundation
> >>> of nginx code." http://www.aosabook.org/en/nginx.html
> >>>
> >>>    But to maximize the use of processors in a multicore environment,
> >>> Nginx do this:
> >>>
> >>>    "nginx doesn't spawn a process or thread for every connection.
> >>> Instead, worker processes accept new requests from a shared "listen"
> >>> socket and execute a highly efficient run-loop inside each worker to
> >>> process thousands of connections per worker"
> >>>
> >>>    My question: There is something similar in Twisted? Or do you
> >>> think that is easy to implement something like that?
> >>>
> >>>    Thanks!
> >>>
> >>> --
> >>> Augusto Mecking Caringi
> We observed really great scaling on multi cores with moving the
> application part either to ampoule for PDF production or in the other
> case I wrote an implementation of self regulating process pool based on
> spread, leaving only the serving to twisted in both cases.
>
> With handing work out to other processes you get another benefit which
> is isolation of python, which is the only way to use a package, like
> reportlab which survives no sort of reentrancy, for a webservice.
>
> Werner
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

Hi Werner

       I want to know whether you have experienced any serious bug with
ampoule? What version of ampoule did you use? Is it 0.2.0? Thanks.

Regards

gelin yan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20120714/f3fca3c5/attachment.html>