[Twisted-Python] spawning tasks after reactor?
darren at ontrenet.com
Wed Dec 31 10:51:38 EST 2008
Thank you for those suggestions. I can post some simple code after I
get further. The psuedo code of what I want to do is like this:
every few seconds, check a message queue (I see how this is done):
if there is a message, spawn a deferred so I don't block:
# The deferred should immediately call a method in a separate
# thread (wisted kind) to process the message
# Allow for 10 or so deferreds to be processing messages
# in parallel while the reactor continues to receive them
# but block scheduling the deferred if the pool is maxed
Sorry if this is lacking in detail, i'm trying to test the idea in a
simple way with twisted first. If I set up a deferred before calling
reactor.run(), then of course, it runs, but after reactor.run() I have
to invoke reactor.callLater(...) so its a bit different, but what I'd
like is something like reactor.callNow(...) that doesn't block and
invokes the generator in a thread pool.
Thanks for any tips! Awesome package.
On Wed, 2008-12-31 at 10:17 -0500, Jean-Paul Calderone wrote:
> On Wed, 31 Dec 2008 09:37:26 -0500, Darren Govoni <darren at ontrenet.com> wrote:
> > I'm new to twisted and have a simple question. I want to run a
> >threaded server that monitors a remote service and spawns
> >queued/threaded tasks as it finds messages.
> Queued/threaded tasks? That do what?
> >I'm having trouble understanding how to do this after a reactor is
> >running since the tasks are not set up before reactor.run(). I see
> >reactor.callLater(). But I want to spawn the task immediately.
> The way to do things after the reactor is running is the same as the way
> to do things before the reactor is running. Just do them. There's no
> difference. If you're having trouble figuring out where to put the code
> that does them, think about when you want the tasks done. You said you
> want to do things when you "find messages". How do you find messages?
> >I tried:
> >but the method is not immediately invoked. It seems its not invoked at
> This will call `self.some_method(object)´ soon. If the call is never
> happening, then you've done something wrong. Perhaps you didn't start
> the reactor, or perhaps the signature of some_method is incompatible
> with that invocation so that when the call was attempted, it just raised
> a TypeError. It's hard to say exactly what's wrong without seeing a
> complete, self-contained, runnable example which demonstrates the
> >I want to create a high-performance threaded scheduler using twist. Any
> Think about ways to not use threads. Anything you can do without threads
> will be higher performance than the threaded equivalent.
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
More information about the Twisted-Python