[Twisted-Python] Synchronization techniques

[glyph at divmod.com]

On 08:32 pm, ellisonbg.net at gmail.com wrote:
>>I am very curious about your 0.00001% case.  Not that I don't believe 
>>such
>>cases exist, but in every case but one (twisted ticket #2545) the 
>>issue has
>>actually been a documentation problem with Twisted, where it wasn't 
>>clear
>>how to do something the "normal" way with Deferreds and such.  I'd 
>>like to
>>know if there is another such doc bug we should be filing :).
>
>The cases I am thinking about is not an example of a doc bug.  The
>most relevant one is related to using Twisted in an interactive python
>(really IPython most of the time) session.  There are two difficulties
>we keep running into:

Hmm.  I hope you're not the guy I talked to at PyCon (I didn't catch 
his/your name) because I'm about to feel very foolish repeating myself 
here, especially if at some point in the interim you responded to this 
and I didn't notice.
>1.  The interactive python interpreter is a completely synchronous
>universe - getting the reactor running in this context seems like a
>hack.  The only way I have seen this done is by running the reactor in
>a different thread.  The problem with this is that it is inevitable
>that you end up wanting to do things with Deferreds in the main thread
>where user code is running.  But, as I understand it, Twisted is not
>thread safe, so at that point, you are playing with (threaded) fire.

IPython's shell is not the same as the standard Python interpreter. 
It's already its own program and therefore has a fair amount of freedom 
in what it does with user input.

Run "python -m twisted/conch/stdio" for an example of an interactive 
session that is held in a non-blocking way.  Not blocking in the code 
*doesn't* mean not blocking for the user - it just means having the 
option not to block.
>2.  Users expect certain things in an interactive python session that
>don't mesh well with Twisted and the asynchronous universe:

This is _exactly_ the documentation issue I was talking about :).
>>>>psi = computeWavefunctionForHydrogen()
>>>>psi.getEnergy(1)
>-13.6
># here the user looks at the energy (a human if statement) and decides
>if they actually want to
># make the following plot.  If the answer were not -13.6, they would
>not make the plot.
>>>>plot(psi.getState(1))
>
>Even if you could get the reactor running in an interactive python
>session it would be crazy to have to write something like (in an
>interactive session):
>>>>d = computeWavefunctionForHydrogen()
>>>>def printAndPlot(psi, n):
>>>>     print psi.getEnergy(n)
>>>>     if abs(psi.getEnergy(n) - (psi.getEnergy(n)) < 1.0e-4:
>>>>         plot(psi.getState(n))
>>>>d.addCallback(printAndPlot, 1)

Here's a screenshot of an interactive session using the command above:

http://twistedmatrix.com/users/glyph/images/content/screenshots/psi- 
energy.png

Unfortunately it doesn't animate, but there is a 1-second pause between 
"<Deferred #0>" and "Deferred #0 called back:".  It's a bit crude since 
it is "deferred-oriented" right now, rather than result oriented, but a 
few clever implementation tricks could easily eliminate the distinction 
(like the "_.result" line).

Is that the kind of user-interaction you want?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20070404/c16895ca/attachment.htm