[Twisted-Python] waitForDeferred Question

[Ed Suominen]

On 3/11/06, Phil Mayers <p.mayers at imperial.ac.uk> wrote:
>> Brian Granger wrote:
>>> I probably should have given more details about what I am trying to
>>> do.  Before I get going, I should mention that I have been using
>>> Twisted heavily for 1.5 years and I have, for the most part, learned
>>> to play the "Twisted Game."
>> If that is the case, you really ought to have known the answer was "you
>> don't" ;o)
>  
> I respectfully disagree that what I am trying to do somehow violates
> the unwritten Twisted ethic.  I am not using threads and I don't have
> blocking code anywhere.  When I write things like:
>  
>>> myresult = a.computeSomethingThatUsesTwisted()
>  
> It does not really block - it only appears to block.  What I mean by
> this is that is does not block execution, it only blocks the user from
> starting new commands.  The full machinery of Twisted is free to
> proceed while the result is pending.

Rather than thinking of your goal as "blocking" the user from starting a
new command when once is pending, think of it as "deferring" execution
of any new commands entered by the user. With that Twisted-compatible
mindset in place, you can design a queue for deferring your user's commands.

If no commands are pending in the queue, you simply run it right away
and return a deferred to its result, as has been discussed. If the user
starts a new command while the first is still pending, simply wrap the
execution of that new command in its own deferred and chain it to the
pending deferred. If the new command depends on the result of the
pending one, that's just fine. When the first command completes, its
result will be passed to the callback that will then run your
second-entered command.

The outcome of all will be pretty cool for your user. He can type "x =
hugeFunction(a)" and get something like "Computing 'x =
hugeFunction(a)...'" Being impatient, he can then type "y =
massiveFunction(x)" and then get "Waiting for x to compute 'y =
hugeFunction(x)...'" and so on. Instead of being forced to just sit and
wait, he can *benefit* from the async power of Twisted by being able to
stack up his computations and see the whole chain of deferred operations.

By the way, I've been down this "it must block" road myself several
times before I eventually realized that it will never work. Code that
sits around and idles while it waits for something can only exist in a
separate thread or process from the Twisted event loop.

Best regards,
Ed Suominen