[Twisted-Python] Re: [Twisted-commits] r14683 - Make thetoMainThread Queue unlimited in size, instead of max size 1.
stephen at thorne.id.au
Fri Oct 7 22:30:50 EDT 2005
On Fri, 7 Oct 2005 13:31:02 -0400, James Y Knight <foom at fuhm.net> wrote:
>On Oct 7, 2005, at 11:05 AM, Stephen Thorne wrote:
>>Make the toMainThread Queue unlimited in size, instead of max size 1.
>>This can cause deadlocks.
>Why does it cause deadlocks? That queue should be the only lock the tsr
>thread blocks upon, besides the GIL. Making the queue infinite means the
>tsr thread could sit there generating large amounts of events before the
>main thread gets around to reading any of them, getting in a state of
>continuous backlog and badness. From what I can see, the select thread
>*should* block until the main thread has read the data.
I'm not exactly sure, but it was deterministic. test_internet.InterfaceTestCase.testWakeUp would cause the reactor to stop dead.
It seems there are significant memory consumption issues, i'm in the process of figuring out if putting a sane upper limit on the queue (suggestions have ranged from 100's to 100,000's) will stop my poor powermac from being brought to its knees by threadedselectreactor.
There are quite a few more failures in threadedselectreactor, I'm seeing non-deterministic behaviour, things like tcp.Client using reactor.callLater(0, self.failIfNotConnected, error) being a problem - because if the fork in the code is taken that goes to that state, self.socket doesn't get defined, and code can end up trying to write to that socket and getting an AttributeError......
Suffice it to say, I'm working on the issues here, and I want my buidslave all green and pretty, I'll see about getting this resolved before the end of the weekend.
More information about the Twisted-Python