[Twisted-Python] Any Message Queue recommendations

[Ross Jekel]

Thanks Brett. Pretty much everything you describe is supposed to be in 
xmlBlaster using the xmlrpc interface. I can give you the start of a 
class that helps access xmlBlaster asynchronously if you want to check 
it out. At this point since I'd just be reinventing xmlBlaster, I'll 
spend a couple of days trying to get the queues working the way I want. 
I basically need the messages to be delivered once and only one to the 
subscriber.  Even if the subscriber was disconnected for a while and 
reconnects later I want the historical messages to be delivered once. 
This isn't rocket science, so I'm surprised it isn't available.

Brett Viren wrote:

>Hi Ross,
>
>Ross Jekel <ross at sourcelabs.com> writes:
>
>  
>
>>I'm writing several service processes that unfortunately need to
>>integrate with Java, PHP, and potentially C/C++ programs (non-legacy).
>>I've been searching for a message queuing layer that would support
>>Python/C++/PHP and Java clients (JMS is not necessarily a requirement). 
>>    
>>
>
>This sounds a lot like what I asked recently regarding a network
>distributed system for producing and consuming "alarms" (basically
>just your "messages").
>
>Consider this:
>
> - XML-RPC for communications.  XML-RPC implementations exist in all
> your target languages.  Twisted has addition support beyond the
> built-in Python XML-RPC.
>
> - Write a central proxy server speaking XML-RPC, (implemented in
> Twisted, of course!).  Every message comes through this proxy.
>
> - Message producers do simple XML-RPC query/response (method call) to
> send a message to the central proxy server.
>
> - Then, depending on how you want your message consumers to behave
> you can either:
>
>  a) have them poll the server.  This can work in two ways.  Either
>  the proxy returns any messages collected since some time, or you can
>  have the query block until a message returns.  Or a mixture of both.
>
>  b) have consumers register with the proxy and provide a callback
>  XML-RPC url, the consumers then go into listen mode and the proxy
>  pushes messages as they come in.
>
>Option (a) is good because your consumers can be behind a firewall.
>It's not so good as it introduces latency and makes the proxy more
>complex as it must handle the buffering of messages.
>
>Option (b) is simpler and introduces less latency between message
>production and consumption but does require your consumers to be able
>to listen on a port and receive incoming connections.  If there is a
>firewall between the consumers and the proxy or if your local network
>overlords don't like programs listening on ports this won't do.
>
>You probably want to also allow for message levels and/or classes so
>consumers can pick and choose what they receive.
>
>And, if you go this route be sure to post your work so I can steal it
>for my project!
>
>Regards,
>-Brett.
>
>_______________________________________________
>Twisted-Python mailing list
>Twisted-Python at twistedmatrix.com
>http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>  
>