[Twisted-Python] spawning tasks after reactor?

Darren Govoni darren at ontrenet.com
Wed Dec 31 10:51:38 EST 2008

Hi Jean-Paul,
  Thank you for those suggestions. I can post some simple code after I
get further. The psuedo code of what I want to do is like this:

every few seconds, check a message queue (I see how this is done):
   if there is a message, spawn a deferred so I don't block:
	# The deferred should immediately call a method in a separate
	# thread (wisted kind) to process the message
	# Allow for 10 or so deferreds to be processing messages
	# in parallel while the reactor continues to receive them
	# but block scheduling the deferred if the pool is maxed
	# out

Sorry if this is lacking in detail, i'm trying to test the idea in a
simple way with twisted first. If I set up a deferred before calling
reactor.run(), then of course, it runs, but after reactor.run() I have
to invoke reactor.callLater(...) so its a bit different, but what I'd
like is something like reactor.callNow(...) that doesn't block and
invokes the generator in a thread pool.

Thanks for any tips! Awesome package.


On Wed, 2008-12-31 at 10:17 -0500, Jean-Paul Calderone wrote:
> On Wed, 31 Dec 2008 09:37:26 -0500, Darren Govoni <darren at ontrenet.com> wrote:
> >Hi,
> >  I'm new to twisted and have a simple question. I want to run a
> >threaded server that monitors a remote service and spawns
> >queued/threaded tasks as it finds messages.
> Queued/threaded tasks?  That do what?
> >
> >I'm having trouble understanding how to do this after a reactor is
> >running since the tasks are not set up before reactor.run(). I see
> >reactor.callLater(). But I want to spawn the task immediately.
> The way to do things after the reactor is running is the same as the way
> to do things before the reactor is running.  Just do them.  There's no
> difference.  If you're having trouble figuring out where to put the code
> that does them, think about when you want the tasks done.  You said you
> want to do things when you "find messages".  How do you find messages?
> >
> >I tried:
> >
> >reactor.callLater(0,self.some_method,object)
> >
> >but the method is not immediately invoked. It seems its not invoked at
> >all.
> This will call `self.some_method(object)´ soon.  If the call is never
> happening, then you've done something wrong.  Perhaps you didn't start
> the reactor, or perhaps the signature of some_method is incompatible
> with that invocation so that when the call was attempted, it just raised
> a TypeError.  It's hard to say exactly what's wrong without seeing a
> complete, self-contained, runnable example which demonstrates the
> behavior.
> >
> >I want to create a high-performance threaded scheduler using twist. Any
> >tips?
> Think about ways to not use threads.  Anything you can do without threads
> will be higher performance than the threaded equivalent.
> Jean-Paul
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

More information about the Twisted-Python mailing list