[Twisted-Python] Re: [Twisted-commits] r14683 - Make the toMainThread Queue unlimited in size, instead of max size 1.

Toby Dickenson tdickenson at devmail.geminidataloggers.co.uk
Mon Oct 10 06:06:24 EDT 2005

On Friday 07 October 2005 18:31, James Y Knight wrote:
> On Oct 7, 2005, at 11:05 AM, Stephen Thorne wrote:
> > Make the toMainThread Queue unlimited in size, instead of max size 1.
> > This can cause deadlocks.
> Why does it cause deadlocks? 

I see one possible deadlock, but Im suprised you dont see other different 
symptoms first:

In the blocking select thread:
1. _doSelectInThread calls _sendToMain, it calls mainWaker to wake up the 
foreign event loop, then returns

In the main thread
2. some application code (?) calls doIteration
3. doIteration calls _sendToThread(_doIterationInThread
4. doIteration calls toMainThread.get(), expecting to block until 'its' 
iteration is complete. However it immediately receives the message sent in 
step 1.
5. doIteration returns
6. The foreign event loop gets round to calling _interleave because of the 
request sent in step 1.
7. _interleave calls toMainThread.get_nowait. This raises a Queue.Empty 
exception, which escapes up to the foreign event loop integration code.

In the blocking select thread

8. _doSelectInThread is called because of the request in step 3.
9. _doSelectInThread calls sendToMain. This never gets removed from the Queue.

At this point:

* The reactor is stalled indefinitely because _interleave raised an exception 
before it called _sendToThread(_doIterationInThread.

* Any subsequent calls to doIteration will cause _doThreadIteration to 
deadlock in sendToMain.

I believe the right solution involves replacing get_nowait with a regular get 
in _interleave.

I guess it is not a problem that doIteration sometimes does not perform a 
whole iteration. There has never been a guarantee about how much work is done 
each iteration of the reactor loop, right?

Toby Dickenson

More information about the Twisted-Python mailing list