[Twisted-Python] multiprocessing capability?

Darren Govoni darren at ontrenet.com
Wed Feb 24 13:11:11 MST 2010


Looks interesting. I'm going to check out that package.


My original request was more along the lines of using Python's new
support for native CPU core's and processes (the multiprocessing package
is for this). Python's built-in thread support has global lock
constraints that underperform in some situations.

But I ran into a problem using multiprocessing module with Twisted that
was pointed out on the Twisted trac with pickling class methods and
apparently Python's CPU threading support attempts to do this in some
situations (e.g. when I try to pass a class method to a native thread).

On Wed, 2010-02-24 at 12:04 -0800, K. Richard Pixley wrote:

> Glyph Lefkowitz wrote: 
> 
> > On Feb 21, 2010, at 8:00 PM, Alexandre Quessy wrote
> > 
> > > Hello everyone,
> > > I have done something similar to this, but I used the children IO
> > > stream to control them. Maybe I should have done that using some
> > > higher level protocol, such as AMP or PB.
> > >     
> > 
> > Using a higher-level protocol is generally better, if for no other reason that it gives you a framework within which to document your design decisions.  It's much easier to say "An AMP command with a 'foo' String argument and a 'bar' Integer argument" than to say "The first two bytes of the message are the length of the first argument.  The next n bytes are the first argument.  The first argument shall be interpreted as... (etc, etc)"
> 
> I'm working on an interface right now to the spread toolkit,
> (http://spread.org), which implements virtual synchrony,
> (http://en.wikipedia.org/wiki/Virtual_synchrony).
> 
> For distributed, symmetric, fault tolerant parallelism in small to
> medium scale with high reliability, this might be an option.
> 
> --rich
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20100224/42cc609b/attachment.html>


More information about the Twisted-Python mailing list