[Twisted-Python] Re: twisted and threading

Paul Moore paul.moore at atosorigin.com
Thu Nov 25 10:24:57 EST 2004

Jp Calderone <exarkun <at> divmod.com> writes:

> twisted-10152003 <at> atlas.lcs.mit.edu wrote:
> > 
> > Ah, but that doesn't work.  As I stated the c library (not mine) calls
> > potentially block forever (or at least longer than I want to wait to
> > shutdown the twisted server).  The thread created by deferTothread may
> > never complete which makes shutting down the server gracefully
> > impossible (or so I thought?).
> > 
>    There is no portable way to terminate a thread without its 
> assistance.  Python supports no API for this - save one, the 
> ridiculously named "setDaemon" Thread method.  Twisted doesn't expose 
> this, nor call it internally, as it can lead to segfaults.
>    Perhaps this should be parameterizable (defaulting to the current 
> behavior, of course), so that poorly behaved libraries can be dealt with?
>    Alternatively, since daemonized threads might lead to this anyway, 
> perhaps you should just add a shutdown event os.kill(os.getpid(), 9) 
> <wwinkink>.

Sorry this is from long ago, possibly the OP has found a solution by now, but I 
thought I'd share my approach in case it was of more general interest...

I have hit the same problem recently (C library calls that might hang, and I 
want to give up and quit regardless in the end). What I came up with was a 
function deferToDaemonThread, which put the call in a daemon thread (one that 
the Python main thread will not wait for). It steals a lot of code from 
deferToThread, but creates a dedicated daemon thread for the call rather than 
using the thread pool.

Hope it's of interest,


from twisted.internet import reactor
from twisted.python import failure
from twisted.internet import defer
from threading import Thread

def deferToDaemonThread(f, *args, **kw):
    """Run function in thread and return result as Deferred."""

    def putResultInDeferred(d, f, args, kw):
	"""Run a function and give results to a Deferred."""
	    result = f(*args, **kw)
	    f = failure.Failure()
	    reactor.callFromThread(d.errback, f)
	    reactor.callFromThread(d.callback, result)

    d = defer.Deferred()
    thread = Thread(target=putResultInDeferred,
	            args=(d, f, args, kw))
    return d

More information about the Twisted-Python mailing list