[Twisted-Python] stop conditions and deferToThread()

[Jean-Paul Calderone]

On Wed, 10 Jun 2009 12:04:16 +0200, Thomas Jakobsen <thomas.jakobsen at alexandra.dk> wrote:
>Hi
>
>As discussed in a previous thread
>
>   http://twistedmatrix.com/pipermail/twisted-python/2009-May/019717.html
>
>a task put in its own thread via deferToThread() won't stop even
>though the reactor has stopped. It has to carry out its own check that
>the reactor is running and stop if that is not the case.

deferToThread is optimized for short-running tasks.  Thread creation costs
are amortized over the lifetime of the process.  This is because the intended
use of deferToThread is to run functions which compute some result and then
return, /not/ for functions which run forever (or at least until the process
is ready to exit).

You should try to find a way to re-arrange your code so that there is no
need to check if the reactor is still running - just do one piece of work
in a thread at a time.  When your non-threaded application code gets the
result back, if no one has asked it to shut down, then it can ask for
another job to be executed.

>My question is: Is it possible to use anything else than the
>reactor.running as stop condition in this way?

Also, I want to point out that while `reactor.running´ isn't private (since
it doesn't start with an underscore, as Twisted's privacy policy dictates it
would need to), it still isn't something you really want to be using.  It's
not part of any interface, which means there is no guarantee it will be
provided by all reactors.

It's also not really for you.  It's a flag that the reactor uses to track
its own internal state.  Its meaning may not actually correspond to the
meaning you'd like to assign to it.  Whenever you're trying to do something
like this (which I still don't recommend), you should create your own state
tracking and use that instead.

Jean-Paul