AW: Re: [Twisted-Python] Can a deferred catch a segmentation fault ?
george at ringdevelopment.com
Fri Sep 21 14:52:31 EDT 2007
On Fri, 2007-09-21 at 10:24 +0100, Phil Mayers wrote:
> On Thu, 2007-09-20 at 11:26 -0700, Ed Suominen wrote:
> > Here I go again touting my AsynQueue package. Sorry, but it just seems
> > to be a very appropriate solution to many of the problems being raised
> > recently.
> Well, to be fair it's an excellent bit of code.
> > I've recently added a "processworker" module that does just what it
> > sounds like. You can now queue up jobs to be run on a separate Python
> > interpreter. If the interpreter crashes due to a segfault or anything
> > else, you just construct a new worker instance and attach it to the
> > queue, and the jobs continue merrily along.
> Interesting. I see for the process worker jobs you pass in a python
> string. One of the conceptual difficulties I've always had with creating
> a farm of subprocesses is ensuring the module import status would be
> valid, so that you could pass a function and class instances across the
> pickle boundary (or whatever)
> Did you consider this approach?
> > In addition to deferred-based priority queuing, the queue object has
> > powerful capabilities for hiring and firing workers, letting workers
> > resign when they can't perform their duties any more, assigning tasks to
> > appropriate workers, and re-assigning tasks from terminated workers.
> > See http://tinyurl.com/349k2o
> > (http://foss.eepatents.com/AsynQueue/browser/projects/AsynQueue/trunk/asynqueue/processworker.py)
> > By the way, AsynQueue (without the new processworker stuff) is now
> > available in Debian testing, thanks to efforts of Eric Evans. Just
> > apt-get install python-asynqueue.
> > Best regards, Ed
> > _______________________________________________
> > Twisted-Python mailing list
> > Twisted-Python at twistedmatrix.com
> > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
More information about the Twisted-Python