[Twisted-Python] Consistent interfaces to asynchronous partially-available services using Deferreds and state machines (was Re: Another approach to allowing __init__ to work with Deferreds)

[Phil Christensen]

On May 12, 2009, at 7:39 AM, Terry Jones wrote:
>> Just as a point of convenience, I would have automatically  
>> determined this
>> list of method names by using a decorator or something.  Having it  
>> as a
>> static list in the method invocation seems to me like it would be  
>> very easy
>> to forget to add or remove a method from the list, and it would  
>> make diffs
>> that touched a user of this class always have two hunks for adding a
>> method; one at the method definition site, one at the call to wrap().
>
> I started out trying to write this using decorators. But I didn't  
> really
> see how to do it. I was using two - one for __init__ and one for the
> wrapped functions. I also tried with decorators and a super class.  
> In the
> end I saw a simple way to do it with the mixin, so went for that.  
> I'd be
> happier with a decorator solution for the reasons you mention.

I don't know if I agree with the need for such a feature (that is,  
deferred __init__ usage), but it was a very interesting coding  
challenge I wanted to take a whack at. I *think* I might have found a  
solution, but I don't know if it falls under the heading of "decorator  
abuse" ;-)

Basically, it requires that the init method set an instance variable  
bound to a Deferred that will fire when the initialization is  
finished. Then, @deferredInit decorators applied to each instance  
method handle checking for and adding callbacks to that original  
"initDeferred".

This way, any method that depends on "complete instantiation" (which  
is probably most or all of them) can have the decorator applied, and  
will have itself added as a callback to the original initDeferred.

Right now, the name of the Deferred used by __init__ is hard coded,  
but you could easily make the decorator take an argument that  
specifies the name to use.

This appears to work for me, but there's a lot of stuff I'm still  
learning about deferreds, and although I read most of this thread, I  
may have missed a use case that won't work in this manner.

Still, it was a fun challenge ;-)

Let me know what you think:


     from twisted.internet import defer, reactor
     from twisted.enterprise import adbapi

     def deferredInit(func):
         if not(hasattr(deferredInit, 'waiting')):
             deferredInit.waiting = {}

         def _deferredInit(self, *args, **kwargs):
             waiting_for_init = self in deferredInit.waiting

             if not(waiting_for_init):
                 if(hasattr(self, 'initDeferred')):
                     deferredInit.waiting[self] = self.initDeferred
                 else:
                     raise RuntimeError("%s doesn't seem to support  
deferred instantion." % self.__class__.__name__)

             def _finish(result):
                 del deferredInit.waiting[self]
                 return func(self, *args, **kwargs)

             def _finish_error(failure):
                 print '_finish_err: %s' % failure

             resultDeferred = defer.Deferred()
             resultDeferred.addCallbacks(_finish, _finish_error)

              
deferredInit.waiting[self].addCallbacks(resultDeferred.callback,  
resultDeferred.errback)

             return resultDeferred

         return _deferredInit

     class TestDeferredInit(object):
         def __init__(self):
             self.pool = adbapi.ConnectionPool("MySQLdb", 'localhost',  
'test', 'test')
             self.initDeferred = self.pool.runQuery("SELECT 'it  
worked';")
             def _finish_init(msg):
                 self.msg = msg
             def _finish_init_error(failure):
                 print '_finish_init_err: %s' % failure
             self.initDeferred.addCallbacks(_finish_init,  
_finish_init_error)

         @deferredInit
         def query(self):
             return self.msg

     if(__name__ == '__main__'):
         def _print(msg):
             print msg
             reactor.stop()

         def _print_error(failure):
             print '_print_err: %s' % failure

         test = TestDeferredInit()

         d = test.query()
         d.addCallbacks(_print, _print_error)

         reactor.run()