[Twisted-Python] Consistent interfaces to asynchronous partially-available services using Deferreds and state machines (was Re: Another approach to allowing __init__ to work with Deferreds)

[Phil Christensen]

On May 14, 2009, at 4:01 AM, Terry Jones wrote:
>>>>>> "Phil" == Phil Christensen <phil at bubblehouse.org> writes:
>
> Phil> I don't know if I agree with the need for such a feature (that  
> is,
> Phil> deferred __init__ usage), but it was a very interesting coding
> Phil> challenge I wanted to take a whack at. I *think* I might have  
> found a
> Phil> solution, but I don't know if it falls under the heading of
> Phil> "decorator abuse" ;-)
>
> Hi Phil
>
> I finally had time to look at your solution a bit (though I've not run
> it). It does a couple of things I wouldn't have thought of, like  
> putting
> the dictionary onto the deferredInit function. A couple of comments,
> supposing I understand your code properly:
>
> - One thing I had hoped to avoid was to slow the class methods down by
>   having them always check the original deferred (or a flag) before  
> taking
>   action. My approach does this by moving them aside and then  
> putting them
>   back in place once the deferred fires. Your solution requires that  
> every
>   decorated method does several extra things before it gets going.  
> That
>   could be greatly reduced if you were to check  
> self.initDeferred.called
>   and simply call the original function if the deferred has fired.

Yeah, I see what you mean. I changed it to directly call the function  
if the initDeferred has already been fired. One catch is that I think  
it's important that the function always return a Deferred, even if  
it's just a succeed() wrapper, so as to provide a consistent interface  
whether __init__ is finished or not.

I added a couple additional calls to my test example to illustrate this.

> - If multiple calls are made to instance methods before the init  
> deferred
>   has fired, they will, as I read it, all try to del
>   deferredInit.waiting[self] in _finish. So I guess that del needs  
> to be
>   conditional or in a try/except.

I realized there's no reason to keep a dictionary anyways, since you  
always have access to `self`. The result means less bookkeeping, which  
is always good...

> - Using self as a key into the dict on initDeferred seems like it
>   addresses Glyph's observation/criticism that my approach raises
>   questions wrt inheritance.

I believe even though I removed that state dictionary, this should  
still work properly, since we always operate on/with `self`.

> - You could use chainDeferred where you're currently using
>   .addCallbacks(resultDeferred.callback, resultDeferred.errback)

Ah yes. I forgot this existed; a lot of my Deferred experience is with  
sequential processes, so I've been using inlineCallbacks for everything.

> That's all for now. I'll see if I have more time to think about all  
> this.
> When I tried to use a decorator the first time, I was also using a  
> super
> class (whereas you're putting state into a dict on the deferredInit
> function) but I got into a mess accessing self properly (partly  
> because, I
> think, I wanted to have a mixin class and I was looking at
> self.__class__.__mro__).
>
> In any case, thanks for replying, for playing with it, and for  
> posting your
> code. I got to learn new things as a result, which is really great :-)

Yeah, same here. It's pretty rare that I can get into any 'semi- 
advanced' discussions on here (things often seem to go between one  
extreme and the other), but I always learn lots when I do.

I'm still not sure if I would use this technique myself, but I'm  
reasonably satisfied with the "scent" of this code. Obviously though,  
I haven't thought through a whole bunch of use cases, but it seems to  
be pretty simple, in the end.

I've attached the revised version of the decorator. Let me know if you  
think of anything else.

Thanks,

-phil

-------------- next part --------------
A non-text attachment was scrubbed...
Name: init_deferred.py
Type: text/x-python-script
Size: 2272 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20090514/71b81de9/attachment.bin 
-------------- next part --------------