[Twisted-Python] Interactive Twisted client (newbie question?)
exarkun at divmod.com
Wed Jan 5 11:15:04 EST 2005
On Wed, 5 Jan 2005 15:44:33 +0000, Jon Blower <jdb at mail.nerc-essc.ac.uk> wrote:
> Thanks very much for your reply, very useful. I've had another think about my
> problem and I guess it boils down to this question: when I'm writing a client
> program, how can I simulate a synchronous function call (avoiding callbacks)
> like this pseudo-Python, which sends a message then returns its reply:
> def sendMessage(message):
> # Wait for reply to arrive
> reply = getReplySomehow()
> return reply
One way is to have getReplySomehow() context switch out of sendMessage()
and into something capable of constructing getReplySomehow()'s return
value, then switching back and returning it to be bound to the name reply.
Twisted has ignore this rather successfully up until recently. It is,
after all, a basic language feature, and should be provided by the runtime.
Nothing about it is Twisted specific, and it is perfectly possible to write
a Twisted program without this trick.
Recently Chris Armstrong's "deferred generator" support was added to
t.i.defer, though. With it, you write code something like this:
from twisted.internet import defer
intermediate = defer.waitForDeferred(getReplySomehow())
reply = intermediate.getResult()
sendMessage = deferredGenerator(sendMessage)
This supposes that getReplySomehow() returns a Deferred, and causes
sendMessage to also return a Deferred (one which fires with reply
eventually). The benefit is not immediately obvious here, since you
are handing reply back to your caller, rather than using it. The exact
equivalent code without this technique is:
However, there is some benefit if, instead of returning reply, sendMessage
uses it locally a bit.
There are other solutions, of course. You can use pre-emptive threads,
as provided by Python's threading module (or Twisted's thin layer on top
of it). Keep in mind that there is pretty much only one threadsafe API in
Twisted (callInThread). If you do this, you will have to be extremely
careful not to call Twisted APIs from the wrong place. There is an example
of a somewhat generic implementation of this solution in my sandbox.
It gives the appearance of working (and may even do so), but consider it
untested and loaded with bugs (I'm certainly not using it in any of my
Greenlets are another solution along these lines. They simply replace
pre-emptive kernel threads with cooperative user-space threads. As such,
you do not need to worry about calling Twisted APIs from the wrong thread,
since there is no wrong thread. Greenlets are implemented with some tricky
voodoo, so once again, while I know of no bugs, I also wouldn't be surprised
if using them caused a rift in space/time to open up and eat your head.
So what's a Python programmer to do? Well, the technique I'll actually
recommend is the simplest: callbacks. Oh, they're a pain to write sometimes,
but they're much less magic than the other solutions described, often perform
better, and all the bugs you'll get when using them are the simple to fix kind,
rather than the my-stack-just-got-swizzled-time-to-segv or the
They also lead to rather more modular code than the synchronous-style equivalents.
You can unit test a callback by itself. How do you unit test lines 7 - 11 of a
As to your wondering about Twisted's utility for clients versus servers, I
think you are close to the truth. The actual divide, though, is concurrency.
If your program only needs to do one thing at a time, Twisted doesn't simplify
much. If your program needs to do two or more things at once, then it does.
Keep in mind that, even for a client, there are often at least two things that
need to be happening: talking to the network and updating the user interface.
Hope this helps,
More information about the Twisted-Python