[Twisted-Python] Twisted-friendly Message bus

[Chaz.]

If you go to the Spread website and read the documentation and source
code it mentions both the number of computer limitations and the
performance hit. Secondly, the system uses a static file to list the
machines that participate in the communication.

First, it won't support more a 200 computers, or so. It appears that
deep in the bowels of the code is a byte sized field! The code is pretty
convoluted, so I never did manage to find out what specifically was the
field they mentioned. If you are going to build a system that involves a
hundred machines or less you are probably alright.

Secondly, as for the performance a search will point out how costly
reliable group communications are in general. I would suggest you load
the software up and run the demo, tracking the messaging passing with
Ethereal. Once this begins to scalable upwards to hundreds of machines
you will see the problem.

Third, the configuration of a Spread cluster is fixed based on a config
file. This was a problem for me; I wanted my system to scale upward by
just adding a machine to the cluster and not having to change a config
file and rebooting the daemon machines! It doesn't make for a very
reliable approach (after all if IM systems could do it why couldn't
non-IM systems?)

It wasn't until I hit the boundaries of the Spread system did I begin to
think about all that was involved in designing a highly reliable, highly
scalable system. In sketching out the design of a reliable group
communication system similar to Spread did I begin to see the weaknesses
in the solution.

Now if you think Spread is the way to go, and it might be for small
clusters of machines, you should do a little research into things like
the SWIM protocol (of which I am doing a twisted implementation right
now and will be releasing to the community) or other approaches to
group-like communication (do a google of overlay networks, which I will
be also releasing). You will find a lot of relevant information. In fact
they will probably lead you to a group at Cornell (and a few people
there) and some people at the U of I at Champaign.

So, as I said Spread is fine for small clusters of machines with a fixed
deployment. It has tolerable performance and overhead. If you find that
you want to scalable upwards of 200 machines or up to thousands, Spread
won't cut it.

For those of you that like reading academic stuff that is grounded in
reality point your browsers to google and find a copy of:

A Scalable Services Architecture
By Tudor Marian, Ken Birman and Robbert van Renesse
All in the Dept of CS at Cornell

(You might recognize the name Birman as the guy that started a lot of
this stuff...his first system was called Isis)

For what it is worth,
Charles "Chaz" Wegrzyn


Paul G wrote:
> 
> ----- Original Message ----- From: "Chaz." <eprparadocs at gmail.com>
> To: "Phil Mayers" <p.mayers at imperial.ac.uk>
> Cc: "Twisted general discussion" <twisted-python at twistedmatrix.com>
> Sent: Monday, June 19, 2006 10:26 PM
> Subject: Re: [Twisted-Python] Twisted-friendly Message bus
> 
> 
>> I've used Spread communication before. While it is a nice system is does
>> have some shortcomings. First, it is a pretty costly system in terms of
>> resources. Secondly, it will only support a hundred or so computers (too
>> small for my application).
> 
> can you elaborate on this please? ideally, i'd like to see some
> reasonably hard data for this. not implying that you're wrong, mind you,
> but i was under a completely different impression wrt resource
> utilization (albeit without extensive firsthand experience).
> 
> -p
>