[Twisted-Python] Re: Using deferreds when writing across unreliable network
Martin Geisler
mg at daimi.au.dk
Fri Oct 10 07:23:46 EDT 2008
"John Aherne" <johnaherne at rocs.co.uk> writes:
Hi John
I have no concrete solution to what you describe, but I can try and
explain how Deferreds work.
> So should my callback or errback fire without me having to initiate
> any action other than setting up the deferred. And should I be using
> a deferred at all for this situation.
First, a Deferred is an amazingly powerful concept, but the
implementation is surpricingly simple. I once posted a simple
explaination for my own project:
http://article.gmane.org/gmane.comp.cryptography.viff.devel/26
The basic idea is that a Deferred is a container for event handling
functions, called callbacks in Twisted. Python has function pointers,
so it is easy to make such a class:
class MiniDeferred:
def __init__(self):
self.callbacks = []
self.value = None
def addCallback(self, func, *extra_args):
self.callbacks.append((func, extra_args))
def callback(self, initial_value):
self.value = initial_value
for func, extra_args in self.callbacks:
self.value = func(self.value, *extra_args)
return self.value
def __str__(self):
if self.value is None:
return "MiniDeferred (no value yet)"
else:
return "MiniDeferred (value: %s)" % (self.value,)
This class skips the error handling (errbacks), but I believe it
illustrates the core ideas of a Deferred. You can use it like this:
>>> x = MiniDeferred()
>>> print x
MiniDeferred (no value yet)
>>> x.addCallback(lambda value: value**3)
>>> x.callback(-3)
>>> print x
MiniDeferred (value: -27)
The crucial thing is that "someone" or "something" must call the
callback method on your Deferred. The general rule to follow is that
it is the code that creates a Deferrred which has the reponsibility of
calling the callback method at the correct time.
So when you use the getPage function you get a Deferred back. Behind
the scenes the getPage function has setup things in such a way that
the callback method of the returned Deferred is called when the page
has arrived -- you must not invole callback yourself in this case.
But if you create a Deferred yourself, then it is up to you to make a
call to the callback method when ready.
And example could be this mini implementation of the DeferredList
class:
class MiniDeferredList(MiniDeferred):
def __init__(self, deferreds):
MiniDeferred.__init__(self)
self.results = [None] * len(deferreds)
self.missing_results = len(deferreds)
for index, deferred in enumerate(deferreds):
deferred.addCallback(self._callback_fired, index)
def _callback_fired(self, result, index):
self.results[index] = result
self.missing_results -= 1
if self.missing_results == 0:
self.callback(self.results)
return result
Here the _callback_fired method invokes the callback method when all
deferreds have fired.
I hope this helps a bit -- it helped me to look at the implementation
in defer.py and see that it was less scary than I had imagined :-)
--
Martin Geisler
VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/.
More information about the Twisted-Python
mailing list