[Twisted-Python] Re: deferToThread - supported alternative to the (deprecated) setTimeout method

Paul Moore pf_moore at yahoo.co.uk
Fri May 26 15:46:27 EDT 2006


Many thanks to all for the helpful responses. They've given me a lot
of options to think about. I'll see where it takes me.

Some specific comments:

On Tue, 23 May 2006 22:43:35 -0400, Jean-Paul Calderone
<exarkun at divmod.com> wrote:

>There's some code along these lines here:
>
>http://twistedmatrix.com/trac/browser/trunk/twisted/internet/base.py#L201

Thanks, that was useful.

>It might make a good example to work from.  Note that any call that
>blocks indefinitely will hold on to a thread indefinitely, and thus
>reduce your effective thread pool size by one.  If this happens
>enough, you'll end up with no free threads in your threadpool, and
>no other threaded tasks will ever be able to complete.

That's a good point. My initial thought was that I'm not writing a
long-running process here (my process runs all the threads, does a bit
of housekeeping when they complete, and then stops) and the blocking
thread condition is rare, so it shouldn't be a problem. But that's
just swapping one rare problem for another, somewhat rarer case. So
maybe I'm better fixing it properly.

On Wed, 24 May 2006 09:03:29 +1000, "Eric Mangold"
<teratorn at twistedmatrix.com> wrote:

>Well this is a bit of a problem. You can't just kill a thread - so it will
>eventually return a result, and Twisted will .callback() its deferred. If
>you've already stepped in, due to a timeout, and .callback()'ed the same
>deferred you'll get an AlreadyCalledError in your log.

Ah. That clarifies why setTimeout is deprecated, and why it's not
appropriate for me. Thanks!

On Wed, 24 May 2006 12:36:37 +1000, Andrew Bennetts
<andrew-twisted at puzzling.org> wrote:

>I don't suppose there's a non-blocking way to do what you want?

Sadly, no. It's a horribly broken API in this respect. (It's the
Oracle database connection API, if you want to know - no timeout
capability, and a rare but definite chance of a complete hang, no
return *ever*).

>Oh -- and remember that while you can't kill threads, you *can* kill processes.
>Consider using subprocesses to do your blocking work.

That's possible, but I'm running on Windows and subprocess management
is not as manageable on that platform (even under Python/twisted). But
it is a thought, and I'll keep it in mind.

Paul.
-- 
The trouble with being punctual is that nobody's there to appreciate
it. -- Franklin P. Jones





More information about the Twisted-Python mailing list