[Twisted-Python] waitForDeferred Question

[Brian Granger]

On 3/11/06, Moof <moof at metamoof.net> wrote:
> > I am not super familiar with PyShell/PyCrust, but I think everything
> > (including the prompt generation) is done asynchronously.  The only
> > thing that could really block would be long running python code
> > executed in the users namespace in PyShell.  But, I do like the idea
> > of having immediate results return directly, but have everything else
> > become a background job.  But, the question still remains - how do you
> > convert a Deferred() to a blocking, directly returned result?
>
> You could use something like
> http://svn.twistedmatrix.com/cvs/sandbox/exarkun/threadwrapper.py?view=markup&rev=12804
> but I really dont' reccommend it. I think redefining the way the
> interpreter works is the way ahead.
>
> One way to do it would be to think of the space between the last >>>
> prompt and the current one as "flexible". How about this for an
> example session:
>
> >>> somethingThatReturnsADeferred()
>
> this will return a deferred, so natually the first thing you do is
> attach callbacks and errbacks to it to know when the deferred has
> finished. You display some indication of this:
>
> <Deferred 1 at 0x12345678. Currently: Running>
> >>>
>
> You even make a nice coloured "running" icon to put on one side, if
> you want. When it stops, you change "Running" to "Completed", in the
> same line. Assuming nothing has been done yet, you either raise the
> exception (or, I'd prefer printing out the traceback and putting the
> Failure in a special variable, called, say _f, it's more flexible) or
> print out repr(d.result) before putting d.result in _ and the deferred
> in, say, d. The important thing here is to print out the result in the
> space between the two >>>, expanding it as necessary. Thus the screen
> ends up looking like this:
>
> >>>  somethingThatReturnsADeferred()
> <Deferred 1 at 0x12345678. Currently: Completed>
> Result from Deferred 1:
> ['this', 'is', 'a', 'result']
> >>>
>
> If someone has already executed a new command, you display *which*
> deferred the result comes form and skip the _ and d assignemnt step,
> maybe assiging it to another special variable.
>

The problem is that the user will want to do:

>>> result = computeSomething()

OR

>>> result = 2.0*computeOneThing()*math.sin(computeAnother())

What should happen in these cases?  Ideally, the _entire_ command
should become a "job" that evaluates in a "smart" manner.  That is,
the final result shouldn't be computed until all other results are
available.  Meanwhile, the prompt should retrn to the user so they can
continue working.  But, I don't think this is possible w/o making
changes to CPython itself.  The alternative is to simply make each
command appear to block.

> Other enhancement would be to add currently pending deferreds to a
> variable or command you can quickly use to check the status, eg:
>
> >>> jobs
> 1. <Deferred 1 at 0x12345678: Running, Next callback: somefunc(result,
> arg1, arg2, kw1=val1>
> 2. <Deferred 2 at 0x12345679: Running, Next callback: somefunc(result,
> arg1, arg2, kw1=val4>
> 3. <Deferred 3 at 0x12345680: Running, Next callback: somefunc(result,
> arg1, arg2, kw1=val3>
> >>> important_deferred = jobs[1] #so that deferred will not be garbage
> collected once it falls out of scope
>
> You might also want to add a convenience function result() which takes
> a deferred and does the whole putting of the result in _ and the
> deferred in d.
>
> There are plenty of ways to do it. Trying to make it look blocking is
> the wrong way to go about it, IMO.
>

For long running commands I completely agree.  But for commands that
complete quickly (unless they fail in which case an exception should
be raised) I disagree.  For example:

>>> a = getPreComputedVariable('a')

Ib this case the onject 'a' on some server needs to be serialized and
brought back to the local machine.  There is no reason to mess with
Deferrreds in this case.

> Also, I'd be quite interested to see a twisted-compatible PyCrust.
> Maybe even adding the twisted log to a tab in the bottom pane of the
> window. I'd be happy to lend a hand in coding such a beast, if you'd
> like one.
>
> It'd be especilly nice if it could do something manhole-compatible, so
> pycrust could attach to remote twisted processes...
>
> Moof
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>


--
Brian Granger
Santa Clara University
ellisonbg at gmail.com