[Twisted-Python] Any Message Queue recommendations
bv at bnl.gov
Thu Jun 9 19:41:52 EDT 2005
Ross Jekel <ross at sourcelabs.com> writes:
> I'm writing several service processes that unfortunately need to
> integrate with Java, PHP, and potentially C/C++ programs (non-legacy).
> I've been searching for a message queuing layer that would support
> Python/C++/PHP and Java clients (JMS is not necessarily a requirement).
This sounds a lot like what I asked recently regarding a network
distributed system for producing and consuming "alarms" (basically
just your "messages").
- XML-RPC for communications. XML-RPC implementations exist in all
your target languages. Twisted has addition support beyond the
built-in Python XML-RPC.
- Write a central proxy server speaking XML-RPC, (implemented in
Twisted, of course!). Every message comes through this proxy.
- Message producers do simple XML-RPC query/response (method call) to
send a message to the central proxy server.
- Then, depending on how you want your message consumers to behave
you can either:
a) have them poll the server. This can work in two ways. Either
the proxy returns any messages collected since some time, or you can
have the query block until a message returns. Or a mixture of both.
b) have consumers register with the proxy and provide a callback
XML-RPC url, the consumers then go into listen mode and the proxy
pushes messages as they come in.
Option (a) is good because your consumers can be behind a firewall.
It's not so good as it introduces latency and makes the proxy more
complex as it must handle the buffering of messages.
Option (b) is simpler and introduces less latency between message
production and consumption but does require your consumers to be able
to listen on a port and receive incoming connections. If there is a
firewall between the consumers and the proxy or if your local network
overlords don't like programs listening on ports this won't do.
You probably want to also allow for message levels and/or classes so
consumers can pick and choose what they receive.
And, if you go this route be sure to post your work so I can steal it
for my project!
More information about the Twisted-Python