[Twisted-Python] Re: Memory size of Deferreds

Michał Pasternak michal.dtz at gmail.com
Tue May 20 14:54:34 EDT 2008


On 2008-05-19, 22:45:26
Martin Geisler <mg at daimi.au.dk> wrote:

> "Michal Pasternak" <michal.dtz at gmail.com> writes:
> 
[...]
> I would love to get some advice on the design of my library -- it is
> the first thing I have made with Twisted. The goal of VIFF is to
> enable shared computations -- think addition and multiplication. 
[...]
>   def add(x, y):
>       sum = gatherResults([x, y])
>       sum.addCallback(lambda results: results[0] + results[1])
>       return sum
> 
> Multiplication is similar, but requires network communication -- that
> is why it returns a Deferred.

I don't think that there's anything wrong with this example in terms of
being written using Twisted Python.

> The important point is
> that as much as possible is done in parallel, that all operations run
> as soon as their operands are available. Twisted made this extremely
> easy thanks to the DeferredList and Deferred abstraction.

This is the point where I think, that you answered your question. If
you do stuff in parallel, you use more Deferreds and more memory. 

I will repeat myself now: I would rather try to find other places to
optimize VIFF, than reducing memory footprint of Deferred. I don't have
a CS degree, I'm not a mathematician, but from what you wrote it seems
that maybe it would be worth to try implementing something stack-based
(like RPN), than the tree approach you seem to be doing now - of course
if having a stack is something you can have using secure multiparty
calculations.

-- 
m




More information about the Twisted-Python mailing list