[Twisted-Python] Multicast XMLRPC

Chaz. eprparadocs at gmail.com
Fri Aug 25 12:05:50 EDT 2006


Samuel Reynolds wrote:
> At 2006-08-24 02:13 PM -0400, you wrote:
>> I have a thousand or more processors running in an application and 
>> need to send out a request to each and every one of them.
> 
> This is an incomplete statement of the problem.
> To get assistance you need to more fully
> state the problem you're trying to solve.
> 
> What are you trying to accomplish?
> 1. Are you...
>    a. sending data *to* each processor from a central server
>       (e.g., configuration data)?
>    b. retrieving data *from* each processor for
>       display/processing at a central server
>       (e.g., status information)?
>    c. both?
> 2. How often do you need to send/receive the data?
> 3. How much latency is acceptable?
> 4. How much data loss is acceptable?
> 
> Your desire to use multicast suggests that you're doing 1a.
> If 1c, you need to specify 2-4 separately for both 1a and 1b.
> 
> Without this (and probably other) information,
> any solution suggested (or adopted) is, at best,
> a hammer looking for a nail.
> 
>> Obviously it is impossible to send it out sequentially and it doesn't 
>> need to.
> 
> Very little is actually impossible.
> 
>> The model of communication I am working from is that most 
>> communications are delivered and only once in a while do they fail 
>> (this is pretty true of an un-congested ethernet-like network).
>>
>> So my algorithm is as follows (and if someone sees something wrong 
>> please let me know...a thousand pairs of eyes and brains is better 
>> than one). Send out the RPC request via Multicast. Set some timeout 
>> (based on  the request type). Since I know the servers in the 
>> collection, when I don't hear from one within the time out, resend the 
>> RPC request.
> 
> Hammer.
> Look at the whole toolbox.
> Don't settle too quickly on one tool;
> you might need a screwdriver, instead.
> 
> For example, each processor might periodically call
> the central server to send its status information.
> Or it might call the central server only when it
> has new data to send, and otherwise make a simpler,
> low-overhead "heartbeat" call for monitoring.
> 
> HTH.
> 
> - Sam
> 
> __________________________________________________________
> Spinward Stars, LLC                        Samuel Reynolds
> Software Consulting and Development           303-805-1446
> http://SpinwardStars.com/            sam at SpinwardStars.com
> 
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 

Since the title of the email is Multicast XML-RPC I would say I was 
pretty specific. I need to send XML-RPC requests to thousands of 
machines, something that can't be done using a connection oriented 
protocol via TCP very efficiently.

Chaz.






More information about the Twisted-Python mailing list