[Twisted-Python] Twisted in a multicore environment
gelin yan
dynamicgl at gmail.com
Sun Jul 15 10:52:05 EDT 2012
On Sun, Jul 15, 2012 at 1:38 AM, Werner Thie <werner at thieprojects.ch> wrote:
> On 7/14/12 12:39 AM, gelin yan wrote:
> >
> >
> > On Wed, Jul 11, 2012 at 2:08 AM, Werner Thie <werner at thieprojects.ch
> > <mailto:werner at thieprojects.ch>> wrote:
> >
> > On 7/10/12 6:17 AM, Laurens Van Houtven wrote:
> > > FWIW, I have used Ampoule to great effect, but as JP points out
> > it's hardly the only option. You're bound to end up with some
> > measure of multiprocessing. Bear in mind that not all workloads are
> > well-suited for this kind of problem! Always measure before deciding
> > to make your codebase that much more complicated :)
> > >
> > >
> > > cheers
> > > lvh
> > >
> > >
> > >
> > > On 10 Jul 2012, at 18:03, exarkun at twistedmatrix.com
> > <mailto:exarkun at twistedmatrix.com> wrote:
> > >
> > >> On 03:14 pm, augustocaringi at gmail.com
> > <mailto:augustocaringi at gmail.com> wrote:
> > >>> Hi,
> > >>>
> > >>> I'm researching the best way to implement/use a Twisted-based
> > >>> server in a multicore environment...
> > >>>
> > >>> There is the Ampoule project, that I realize is considered
> the
> > >>> best way to do that. Right?
> > >>
> > >> It's a way. "Best" depends on the details and goals of the
> project.
> > >>
> > >> Here's a stackoverflow question/answer on basically the same
> > topic. In
> > >> particular, it specifically answers the question of a listening
> port
> > >> shared between multiple processes and gives examples of how to
> > do this:
> > >>
> > >> http://bit.ly/MiCHtQ
> > >>
> > >> Jean-Paul
> > >>> I'm also reading about the internals of Nginx HTTP server.
> This
> > >>> server utilizes the same reactor pattern of Twisted (epoll
> > based)...
> > >>>
> > >>> "What resulted is a modular, event-driven, asynchronous,
> > >>> single-threaded, non-blocking architecture which became the
> > foundation
> > >>> of nginx code." http://www.aosabook.org/en/nginx.html
> > >>>
> > >>> But to maximize the use of processors in a multicore
> > environment,
> > >>> Nginx do this:
> > >>>
> > >>> "nginx doesn't spawn a process or thread for every
> connection.
> > >>> Instead, worker processes accept new requests from a shared
> > "listen"
> > >>> socket and execute a highly efficient run-loop inside each
> > worker to
> > >>> process thousands of connections per worker"
> > >>>
> > >>> My question: There is something similar in Twisted? Or do you
> > >>> think that is easy to implement something like that?
> > >>>
> > >>> Thanks!
> > >>>
> > >>> --
> > >>> Augusto Mecking Caringi
> > We observed really great scaling on multi cores with moving the
> > application part either to ampoule for PDF production or in the other
> > case I wrote an implementation of self regulating process pool based
> on
> > spread, leaving only the serving to twisted in both cases.
> >
> > With handing work out to other processes you get another benefit
> which
> > is isolation of python, which is the only way to use a package, like
> > reportlab which survives no sort of reentrancy, for a webservice.
> >
> > Werner
> >
> > _______________________________________________
> > Twisted-Python mailing list
> > Twisted-Python at twistedmatrix.com
> > <mailto:Twisted-Python at twistedmatrix.com>
> > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> >
> >
> > Hi Werner
> >
> > I want to know whether you have experienced any serious bug with
> > ampoule? What version of ampoule did you use? Is it 0.2.0? Thanks.
>
> Hi gelin
>
> Python 2.7.1 Stackless 3.1b3 060516 (release27-maint, Aug 9 2011,
> 02:32:21)
> [GCC 4.2.1 20070719 [FreeBSD]] on freebsd8
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import ampoule
> >>> ampoule.__version__
> '0.2.0'
> >>> import twisted
> >>> twisted.__version__
> '12.0.0'
>
> sitting in production now for almost six month, a webservice for
> template based PDF production with reportlab & z3c.rml.
>
> The only maintenance I do is creating more templates, never done a
> restart so far, logs are clean.
>
> As painless as one could wish for
>
> Werner
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
Hi Werner
Thanks for your valuable info. It looks like you are using stackless
python with twisted. Have you taken any benefit from both of them?
Stackless is awesome but I have no idea what is the better way to work with
twisted. Thanks.
Regards
gelin yan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20120715/02fdfb35/attachment-0001.htm
More information about the Twisted-Python
mailing list