[Twisted-Python] deferToThread and reactor loop

exarkun at twistedmatrix.com exarkun at twistedmatrix.com
Thu Nov 1 18:03:31 EDT 2012

On 29 Oct, 05:58 pm, tobias.oberstein at tavendo.de wrote:
>I've got a question regarding
>deferToThread / runWithConnection
>I have a network server that accepts RPCs and forwards those to a 
>relational database calling stored procedures.
>The call of the stored procedures is happening via "runWithConnection" 
>.. that is on a background thread pool (since that DBs driver is 
>Now, when there is a large number of RPCs coming in roughly at once, 
>the latency of each call forwarded to the RDBMs increased into the 10's 
>of ms.
>I have created a self-contained test program without DB etc that I 
>think reproduces the "issue":
>Am I correct that "deferToThread" does not immediately forward the call 
>to a background thread, but only the next time the reactor loop runs?

Hi Tobias,

Sorry, I haven't had a chance to read your minimal example in any 
detail.  However, I can direct you to the implementation of 


Notice the `self.q.put(o)`.  This matches up with the call to 
`self.q.get` in the same module:


Together, these bits of source should demonstrate that there's no 
waiting for a reactor iteration before the work is enqueued.  The work 
goes into the Queue instance, and instantly any worker thread is free to 
grab it.


More information about the Twisted-Python mailing list