[Twisted-Python] Re: [Twisted-commits] r14683 - Make the toMainThread Queue unlimited in size, instead of max size 1.

[Toby Dickenson]

On Monday 10 October 2005 11:06, Toby Dickenson wrote:
> On Friday 07 October 2005 18:31, James Y Knight wrote:
> > 
> > On Oct 7, 2005, at 11:05 AM, Stephen Thorne wrote:
> > > Make the toMainThread Queue unlimited in size, instead of max size 1.
> > > This can cause deadlocks.
> > 
> > Why does it cause deadlocks? 
> 
> I see one possible deadlock, but Im suprised you dont see other different 
> symptoms first:
> 
> In the blocking select thread:
> 1. _doSelectInThread calls _sendToMain, it calls mainWaker to wake up the 
> foreign event loop, then returns
> 
> In the main thread
> 2. some application code (?) calls doIteration
> 3. doIteration calls _sendToThread(_doIterationInThread
> 4. doIteration calls toMainThread.get(), expecting to block until 'its' 
> iteration is complete. However it immediately receives the message sent in 
> step 1.
> 5. doIteration returns
> 6. The foreign event loop gets round to calling _interleave because of the 
> request sent in step 1.

Hmmmm, no need for anything quite so elaborate above. get_nowait is documented 
to raise the Queue.Empty exception (which leads to deadlock in tsr) if the 
Queue is "Empty or Locked". 

http://www.python.org/doc/2.3.5/lib/module-Queue.html

Im sure that get_nowait needs to be replaced with a regular blocking get, as I 
described below.

> 7. _interleave calls toMainThread.get_nowait. This raises a Queue.Empty 
> exception, which escapes up to the foreign event loop integration code.
> 
> In the blocking select thread
> 
> 8. _doSelectInThread is called because of the request in step 3.
> 9. _doSelectInThread calls sendToMain. This never gets removed from the 
Queue.
> 
> At this point:
> 
> * The reactor is stalled indefinitely because _interleave raised an 
exception 
> before it called _sendToThread(_doIterationInThread.
> 
> * Any subsequent calls to doIteration will cause _doThreadIteration to 
> deadlock in sendToMain.
> 
> 
> I believe the right solution involves replacing get_nowait with a regular 
get 
> in _interleave.
> 
> I guess it is not a problem that doIteration sometimes does not perform a 
> whole iteration. There has never been a guarantee about how much work is 
done 
> each iteration of the reactor loop, right?
> 
> 
> 
> -- 
> Toby Dickenson
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> 

-- 
Toby Dickenson