[Twisted-Python] daemon thread support

Andrew Bennetts andrew-twisted at puzzling.org
Tue Dec 2 19:26:40 EST 2003

On Tue, Dec 02, 2003 at 01:28:36PM -0500, Itamar Shtull-Trauring wrote:
> On 02 Dec 2003 17:55:05 +0000
> Martin Waite <martin at datacash.com> wrote:
> > Everything works fine, but I cannot get the application
> > to close down cleanly - it always blocks waiting for the 
> > database threads to stop.
> Instead of having a reader thread, have read tasks that you send to
> thread. That is, don't send one function to callInThread that does a
> while loop, send functions that do one task and then exit.
> Also look at twisted.enterprise.adbapi and the enterprise howto.

Alternatively, have some way to signal to your threads that they should
shutdown cleanly.  I'm assuming your thread does something like:

    while 1:
        nextTask = queue.get()  # queue is a Queue.Queue instance

You could change it to:

    while 1:
        nextTask = queue.get()  # queue is a Queue.Queue instance
        if nextTask is None:

And then all you need to do is make sure you add a None to your threads'
queues when you shutdown, e.g. from stopService.  I've used this method in
the past to shutdown worker threads -- I didn't want them to finish
prematurely, because otherwise there was a risk that the data they were
writing to the DB would get lost.  So I used this sort of explicit
notification, rather than Python's thread.setDaemonic(1).

For reader threads, obviously data loss isn't a problem, so you could just
set those to be daemonic -- although I was creating my threads myself,
rather than using callInThread.  If you use callInThread, do it the way
Itamar suggests: send functions that do their task then exit.  callInThread
maintains a threadpool to service those functions quickly (with a maximum
size to prevent too many running at once).  Otherwise, if you want to do
your own long-running thread, just create it yourself, and you can
setDaemonic on it if appropriate.


More information about the Twisted-Python mailing list