[Twisted-Python] Re: [Twisted-commits] r14683 - Make thetoMainThread Queue unlimited in size, instead of max size 1.

Stephen Thorne stephen at thorne.id.au
Fri Oct 7 22:30:50 EDT 2005


On Fri, 7 Oct 2005 13:31:02 -0400, James Y Knight <foom at fuhm.net> wrote:
>
>On Oct 7, 2005, at 11:05 AM, Stephen Thorne wrote:
>>Make the toMainThread Queue unlimited in size, instead of max size 1.
>>This can cause deadlocks.
>
>Why does it cause deadlocks? That queue should be the only lock the  tsr 
>thread blocks upon, besides the GIL. Making the queue infinite  means the 
>tsr thread could sit there generating large amounts of  events before the 
>main thread gets around to reading any of them,  getting in a state of 
>continuous backlog and badness. From what I can  see, the select thread 
>*should* block until the main thread has read  the data.

I'm not exactly sure, but it was deterministic. test_internet.InterfaceTestCase.testWakeUp would cause the reactor to stop dead.

It seems there are significant memory consumption issues, i'm in the process of figuring out if putting a sane upper limit on the queue (suggestions have ranged from 100's to 100,000's) will stop my poor powermac from being brought to its knees by threadedselectreactor.

There are quite a few more failures in threadedselectreactor, I'm seeing non-deterministic behaviour, things like tcp.Client using reactor.callLater(0, self.failIfNotConnected, error) being a problem - because if the fork in the code is taken that goes to that state, self.socket doesn't get defined, and code can end up trying to write to that socket and getting an AttributeError......

Suffice it to say, I'm working on the issues here, and I want my buidslave all green and pretty, I'll see about getting this resolved before the end of the weekend.

Stephen.




More information about the Twisted-Python mailing list