[Twisted-Python] How to "monitor" parallel executions of Deferreds

[Pedro Sanchez]

I already tested DeferredList and it does just what I was looking for.

There was another suggestion to use defer.gatherResults() for which I
didn't find any documentation. I have yet to browse the code to see what
it does.

Thanks for the reply. 

-- 
Pedro



On Thu, 2005-25-08 at 18:00 -0400, Jason Fritcher wrote:
> Pedro Sanchez wrote:
> > Hello,
> > 
> > I need to implement a function (say, myfunc) that fires several
> > independent Deferreds (using deferToThread), and that returns a deferred
> > itself. A client of this function (say, client) is expected to do
> > something like this:
> > 
> > def client():
> >     def done(success)
> >         if success:
> >            print 'd1 and d2 are done'
> > 
> >     d = myfunc()
> >     d.addCallback(done)
> >     ...
> > 
> > 
> > myfuct() above does something like this:
> > 
> > def myfunc():
> >     def checkFunc1(success):
> >     ...
> > 
> >     def checkFunc2(success):
> >     ...
> > 
> >     d = defer.Deferred()
> >     d1 = deferToThread(func1).addCallback(checkFunc1)
> >     d2 = deferToThread(func2).addCallback(checkFunc2)
> >     d.addCallback(????)
> > 
> >     return d
> > 
> > Clearly d, d1, and d2 in the above are disconnected. And I don't want to
> > serialize d1 and d2 which would be a way to connect them all.
> > 
> > The problem is I don't know how d should be defined. The idea is that
> > the 'done' function in the client function is only called when both
> > threads d1 and d2 are done.
> 
> I think what you're looking for is DeferredList. You pass it a list of
> deferreds you want to wait on and DeferredList will only fire its
> callback method once all of the other deferred are complete. Read the
> docs because the first callback you add to it will get a list of results
> from all the defereds.
> 
> So for your above code, you'd want to do something like...
> 
> d1 = deferToThread(...)
> d2 = deferToThread(...)
> dl = defer.DeferredList([d1, d2])
> dl.addCallback(...) # If needed here.
> return dl
> 
> Just be aware to not add any additional callbacks to d1 or d2 after you
> add them to the deferredlist.
> 
> Hope this helps. :)
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python