[Twisted-Python] deferToThread and reactor loop

Tobias Oberstein tobias.oberstein at tavendo.de
Fri Nov 2 12:15:33 EDT 2012


Hi Jean-Paul,

> >**************
> >Am I correct that "deferToThread" does not immediately forward the call
> >to a background thread, but only the next time the reactor loop runs?
> >**************

> However, I can direct you to the implementation of
> deferToThread:
> 
> http://twistedmatrix.com/trac/browser/trunk/twisted/python/threadpool.py#L1
> 19
> 
> Notice the `self.q.put(o)`.  This matches up with the call to `self.q.get` in the
> same module:
> 
> http://twistedmatrix.com/trac/browser/trunk/twisted/python/threadpool.py#L1
> 58
> 
> Together, these bits of source should demonstrate that there's no waiting for a
> reactor iteration before the work is enqueued.  The work goes into the Queue
> instance, and instantly any worker thread is free to grab it.

Ok, I see.

There might be a mutex or something in the Queue implementation (if it's not
a lockless queue implementation) or the GIL might be involved.

I have no convinving explanation for the behavior I see (which also seems to be
platform agnostic).

Another wild guess (besides above) I have: maybe the OS does not immediately
schedule other process threads for execution if the current thread (the one
pushing to the Queue) is very busy ..

Thanks,
Tobias




More information about the Twisted-Python mailing list