[Twisted-Python] daemon thread support
Andrew Bennetts
andrew-twisted at puzzling.org
Tue Dec 2 19:26:40 EST 2003
On Tue, Dec 02, 2003 at 01:28:36PM -0500, Itamar Shtull-Trauring wrote:
> On 02 Dec 2003 17:55:05 +0000
> Martin Waite <martin at datacash.com> wrote:
>
> > Everything works fine, but I cannot get the application
> > to close down cleanly - it always blocks waiting for the
> > database threads to stop.
>
> Instead of having a reader thread, have read tasks that you send to
> thread. That is, don't send one function to callInThread that does a
> while loop, send functions that do one task and then exit.
>
> Also look at twisted.enterprise.adbapi and the enterprise howto.
Alternatively, have some way to signal to your threads that they should
shutdown cleanly. I'm assuming your thread does something like:
while 1:
nextTask = queue.get() # queue is a Queue.Queue instance
process(nextTask)
You could change it to:
while 1:
nextTask = queue.get() # queue is a Queue.Queue instance
if nextTask is None:
break
process(nextTask)
And then all you need to do is make sure you add a None to your threads'
queues when you shutdown, e.g. from stopService. I've used this method in
the past to shutdown worker threads -- I didn't want them to finish
prematurely, because otherwise there was a risk that the data they were
writing to the DB would get lost. So I used this sort of explicit
notification, rather than Python's thread.setDaemonic(1).
For reader threads, obviously data loss isn't a problem, so you could just
set those to be daemonic -- although I was creating my threads myself,
rather than using callInThread. If you use callInThread, do it the way
Itamar suggests: send functions that do their task then exit. callInThread
maintains a threadpool to service those functions quickly (with a maximum
size to prevent too many running at once). Otherwise, if you want to do
your own long-running thread, just create it yourself, and you can
setDaemonic on it if appropriate.
-Andrew.
More information about the Twisted-Python
mailing list