[Twisted-web] Reactor in a multithreaded program

[Ellers]

> Are you sure reactor.callFromThread(reactor.stop) is being called?

Yes and no - I was sure it was being called (a print statement before) BUT
it turns out it was actually calling and returning ok.

But I found the problem! I needed this at the start:

from twisted.python import threadable
threadable.init( with_threads=1)

DOH!

> And make sure you aren't accidentally doing
> "reactor.callFromThread(reactor.stop())",
> which is a common mistake!

When I first read this I thought "damn thats what I've done" but I wasn't.

>> Is there an advised pattern for working with the reactor with multiple
>> threads?
>
> Yes: use callFromThread everytime you need to call a Twisted function
from a thread :)

;)

The following is more what I meant by a discussion of patterns. (These are
NOT real.. just designed to illustrate for discussion):

  The wxwidgets pattern: simple case is to use the wxreactor.
  However, you COULD run the GUI normally and run the reactor in a
non-main thread. Be sure to communicate with the reactor using
callFromThread or having a protocol reading a Queue.Queue() object.

  The readline pattern: you have a shell-like interface and need to call
the blocking function raw_input(). (Sure, you can use the stdin protocol
in normal twisted but that won't play with readline).

  You run the reactor in a non-main thread and communicate with it as above.


And the pattern could come with a simple proof-of-concept program, like
this one...

I also have a readline example working now. I can post it to the list if
anyone is interested.


>8 --- >8 ---

#!/bin/env python

"""Trivial demo - demonstrate the reactor in a NON main thread.

You can telnet to port 7001 and see echos of everything you type for
10 seconds. After that it should close cleanly. But it doesn't - yet.
"""

import sys
import time
import threading
from twisted.internet import reactor
from twisted.protocols import wire
from twisted.internet.protocol import Factory
from twisted.python import threadable

threadable.init( with_threads=1)

class ReactorFacade:

	def __init__( self ):
		self.reactorThread = threading.Thread(
			target=reactor.run,
			name="ReactorThread",
			kwargs={'installSignalHandlers':0},
			)

	def start(self):
		# assume not yet started
		print "Facade: starting..."
		self.reactorThread.start()
		print "Facade: started ok"

	def stop(self):
		# assume already running
		print "Facade: request stop"
		reactor.callFromThread(reactor.stop)
		print "Facade: request stopped ok"
		self.reactorThread.join()

factory = Factory()
factory.protocol = wire.Echo
reactor.listenTCP( 7001, factory ) # <-- you don't NEED a protocol, but
for example...
print "Echo on 7001"
facade = ReactorFacade()
print "Main: Starting..."
facade.start()
print "Main: Sleeping..."
time.sleep(10)
print "Main: Stopping..."
facade.stop()
print "Main: Finished" # <-- will hang here

>8 --- >8 ---