[Twisted-Python] Multicast XMLRPC

[Samuel Reynolds]

At 2006-08-24 02:13 PM -0400, you wrote:
>I have a thousand or more processors running in an application and need to 
>send out a request to each and every one of them.

This is an incomplete statement of the problem.
To get assistance you need to more fully
state the problem you're trying to solve.

What are you trying to accomplish?
1. Are you...
    a. sending data *to* each processor from a central server
       (e.g., configuration data)?
    b. retrieving data *from* each processor for
       display/processing at a central server
       (e.g., status information)?
    c. both?
2. How often do you need to send/receive the data?
3. How much latency is acceptable?
4. How much data loss is acceptable?

Your desire to use multicast suggests that you're doing 1a.
If 1c, you need to specify 2-4 separately for both 1a and 1b.

Without this (and probably other) information,
any solution suggested (or adopted) is, at best,
a hammer looking for a nail.

>Obviously it is impossible to send it out sequentially and it doesn't need to.

Very little is actually impossible.

>The model of communication I am working from is that most communications 
>are delivered and only once in a while do they fail (this is pretty true 
>of an un-congested ethernet-like network).
>
>So my algorithm is as follows (and if someone sees something wrong please 
>let me know...a thousand pairs of eyes and brains is better than one). 
>Send out the RPC request via Multicast. Set some timeout (based on  the 
>request type). Since I know the servers in the collection, when I don't 
>hear from one within the time out, resend the RPC request.

Hammer.
Look at the whole toolbox.
Don't settle too quickly on one tool;
you might need a screwdriver, instead.

For example, each processor might periodically call
the central server to send its status information.
Or it might call the central server only when it
has new data to send, and otherwise make a simpler,
low-overhead "heartbeat" call for monitoring.

HTH.

- Sam

__________________________________________________________
Spinward Stars, LLC                        Samuel Reynolds
Software Consulting and Development           303-805-1446
http://SpinwardStars.com/            sam at SpinwardStars.com