[Twisted-Python] Question on deferreds

[Phil Mayers]

Frank Millman wrote:

> I have been experimenting with using a socket client instead of Twisted, and
> I got something up and running quite quickly. When I take a step back and
> look at how I got it working, I feel that I should be able to apply the same
> technique to Twisted, but I cannot get it to work. I will show both methods,
> and perhaps someone can point me in the right direction.

No, of course you can't apply the same technique. A socket client is a 
blocking, synchronous bit of code. The ENTIRE POINT of Twisted is that 
all operations are non-blocking, and asynchronous. Basically you're 
going to find it hard going until that's clear (I know I did).

And sadly you're using wx, which is REALLY BAD and does not interact 
well with Twisted until recent developments (see below).

> 
> The trick is that I have two threads running concurrently - a wxPython main
> loop, and a subthread that monitors the socket. The socket loop looks like
> this (simplified), subclassed from threading.Thread -


> 
>     def run(self):
>         readable = [s.fileno()]
>         error = []
>         self.sendData = []
>         while 1:
>             if self.sendData:
>                 writable = [s.fileno()]
>             else:
>                 writable = []
>             r,w,e = select.select(readable,writable,error,0.01)
>             if r:
>                 self.recvData = s.recv(1024)
>             if w:
>                 s.send(self.sendData.pop(0))
> 
>     def checkData(self,item,value):  # this is called from the *wx* thread
>         self.recvData = None
>         self.sendData.append(cPickle.dumps((CHECK,item,value)))
>         while self.recvData is None:
>             sleep(0.01)
>         return self.recvData
> 
> As you can see, checkData() blocks the wxPython thread until it receives a
> response from the socket thread.

Fine. Odd, but fine (I'd use a semaphore as opposed to while True: sleep 
but whatever floats your boat)

> 
> This is how I tried with Twisted -
> 
>     def checkData(self,item,value):  # this is called from the *wx* thread
>         self.recvData = None
>         self.callCheckData(item,value)
>         while self.recvData is None:
>             sleep(0.01)
>         return self.recvData
> 

Bzzt! Error.

You cannot call twisted functions from a thread. See extensive (and 
repetitive) posts in the archives about "threadselectreactor AND wx" for 
about a million posts describing this.

Other than that, and the fact you've got no locking around mutable data 
structures, that code would probably work.

>     def callCheckData(self,item,value):
>  
> self.avatar.callRemote('checkData',item,value).addCallback(self.dataChecked)
> 
>     def dataChecked(self,answer):
>         self.recvData = answer
> 
> It looks as if it should work, but the server method 'perspective_checkData'
> never gets called.

It should not and does not work for the reason above.

I believe "threadselectreactor" is what you want for wx integation. 
Sadly, I've never used threadselectreactor or wx, so can't be more 
useful to you.

See:

http://bob.pythonmac.org/archives/2005/04/17/twisted-and-foreign-event-loops/
http://twistedmatrix.com/pipermail/twisted-python/2005-April/thread.html#10146