[Twisted-Python] DeferredDict, defaultdict, map/reduce, silliness

Ed Suominen general at eepatents.com
Tue Nov 4 12:35:19 EST 2008


Sorry about the dupe, but the link I sent in the earlier post was for a 
more general name:item class. My persistent async dictionary is built on 
that:

http://foss.eepatents.com/api/sAsync/sasync.pdict.PersistentDict.html

--------------------------------------------------------------------------

Terry,

I wrote something like what you're talking about for sAsync. It is a
database-persisted store for named items.

http://foss.eepatents.com/api/sAsync/sasync.items.Items.html

Best regards, Ed

Terry Jones wrote:
> This isn't meant to be taken too seriously - perhaps just food for someone
> else to chew on while trying to relax...
> 
> I was thinking the other night about what a DeferredDict might look like,
> what it might be useful for, if anything, etc. We have DeferredList of
> course, so why not a dict?  Then I also remembered collections.defaultdict
> and couldn't resist the appeal of trying defaultdict(defer.Deferred).  That
> thinking led to the map/reduce silliness below. I don't know that it leads
> anywhere - I was just playing around.
> 
> Terry
> 
> 
> import sys
> from collections import defaultdict
> from twisted.internet import defer, reactor
> 
> class MapReduce(object):
>     def __init__(self, init, cb):
>         self.init = init
>         self.cb = cb
>         self.d = defaultdict(defer.Deferred)
> 
>     def map(self, *keys):
>         for key in keys:
>             self.d[key].addCallback(self.cb)
> 
>     def reduce(self):
>         def finalize(result, key):
>             return (key, result)
>         deferreds = []
>         for key, d in self.d.iteritems():
>             d.addCallback(finalize, key)
>             deferreds.append(d)
>             d.callback(self.init)
>         return defer.gatherResults(deferreds)
> 
> def main():
>     def printResult(result):
>         print result
>         reactor.stop()
>         
>     mr = MapReduce(0, lambda x: x + 1)
>     for line in sys.stdin.readlines():
>         mr.map(*line.split())
>     d = mr.reduce().addCallback(printResult)
>     
> if __name__ == '__main__':
>     reactor.callLater(0, main)
>     reactor.run()
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> 
> 





More information about the Twisted-Python mailing list