[Twisted-Python] spawning tasks after reactor?

Jean-Paul Calderone exarkun at divmod.com
Wed Dec 31 11:16:59 EST 2008


On Wed, 31 Dec 2008 10:51:38 -0500, Darren Govoni <darren at ontrenet.com> wrote:
>Hi Jean-Paul,
>  Thank you for those suggestions. I can post some simple code after I
>get further. The psuedo code of what I want to do is like this:
>every few seconds, check a message queue (I see how this is done):
>   if there is a message, spawn a deferred so I don't block:
>	# The deferred should immediately call a method in a separate
>	# thread (wisted kind) to process the message
>	# Allow for 10 or so deferreds to be processing messages
>	# in parallel while the reactor continues to receive them
>	# but block scheduling the deferred if the pool is maxed
>	# out

A Deferred is just a convenient way to keep track of callbacks.  It's
not an active thing, so to "spawn" one doesn't make much sense.  Instead,
you spawn something else and use a Deferred to keep track of its result.

>
>Sorry if this is lacking in detail, i'm trying to test the idea in a
>simple way with twisted first. If I set up a deferred before calling
>reactor.run(), then of course, it runs, but after reactor.run() I have
>to invoke reactor.callLater(...) so its a bit different, but what I'd
>like is something like reactor.callNow(...) that doesn't block and
>invokes the generator in a thread pool.

You probably want twisted.internet.threads.deferToThreadPool (or the older
deferToThread).  From its docstring:

  Call the function C{f} using a thread from the given threadpool and return
  the result as a Deferred.

Jean-Paul




More information about the Twisted-Python mailing list