[Twisted-Python] Using Twisted with MQ
arkanes at gmail.com
Sat Jun 9 04:20:06 EDT 2007
On 6/8/07, David Mitchell <monch1962 at gmail.com> wrote:
> Hello all,
> I've got what I think is a potential application for Twisted, but I'm
> having trouble figuring out whether Twisted is going to be the best
> path or not. I've worked through the Twisted Finger tutorial and read
> through the "Ball of snakes" O'Reilly book, but I'm not sure I've got
> my head around Twisted sufficiently.
> I have to simulate a server that reads messages off IBM's MQ queues,
> and posts responses back to different MQ queues. I've used both pymqi
> and Python/COM (on Windows only) to perform this sort of task in the
> past, but now I need something that's going to be able to respond to
> incoming MQ messages at a very high rate.
> I suspect I'm going to have to use reactor.callInThread to spawn off
> threads within Twisted in order to handle IO with the MQ queues, as
> I'm not aware of any event-based interface to MQ and therefore any
> reads/writes from queues are potentially going to block for a while.
> Is Twisted suited for this sort of application? Better yet, is anyone
> aware of info I can access regarding this sort of application with
> Twisted? - Google hasn't been much help...
> Thanks in advance
> Dave Mitchell
I haven't written any real code that does this, but I was just
investigating it for feasability myself a couple days ago. I wrote a
thread that looked basically like this:
(pseudocode, as I don't have any of the code in front of me)
res = mq.get() #blocking call, with a fairly long timeout - 5-10 seconds
except Timeout: #pymqi will translate the timeout to an exception
if self.exit.is_set(): #threading.Event to check for termination.
This just spins happily along, and dispatches processing into the
Twisted event loop. I didn't really test it for performance or
latency. Used reactor.callInThread to dispatch the writes.
More information about the Twisted-Python