[Twisted-Python] Running callbacks in a non-main thread
Phil Mayers
p.mayers at imperial.ac.uk
Fri Feb 3 17:31:19 MST 2006
Gabe Rudy wrote:
> Hey,
> I realize that the point of deferreds is to have process/network intensive
> tasks work on in their own thread and send the results to callback function
No. The exact opposite in fact. The purpose of a deferred is to act as a
placeholder for a result that will be available later, and to call
callbacks or errbacks that receive that result, in a single threaded
event-driven system.
> in the main thread, but is there away to have a deferred process it's
> callback in a non-main thread?
If I understand you, the deferred callback will take a long time. Your
code is indicative of a misunderstanding of deferreds, which you need
not feel bad about - they're not an easy concept at first.
Broadly, if you have code that looks like this under synchronous frameworks:
def operation():
res = blockingCall()
# the following function takes time
res2 = doCalc(res)
return res2.field
...that needs to be turned into:
def asyncOperation():
deferred = asyncCall()
deferred.addCallbacks(asyncOperation_2, log.err)
return deferred
def asyncOperation_2(value):
# doCalc is still expensive
deferred = deferToThread(doCalc, value)
deferred.addCallbacks(asyncOperation_3, log.err)
# by returning a deferred, the deferred that calls us will be
# paused until this "INNER" deferred finishes, so it all "just works"
return deferred
def asyncOperation_3(doCalc_result):
# this is a trivial reformatting operation
return doCalc_result.field
More information about the Twisted-Python
mailing list