[Twisted-Python] pushing out same message on 100k TCPs

Phil Mayers p.mayers at imperial.ac.uk
Sat Feb 11 05:48:45 EST 2012

On 02/10/2012 09:54 PM, Glyph Lefkowitz wrote:

> I believe the best model for this kind of high-volume
> reliable-multicast-over-unicast is a spanning tree, like what IRC

For what it's worth, real IP multicast is quite commonly used for 
distributing short messages to many clients in realtime in some closed 
networks, such as financial trading systems. With good network equipment 
that can handle low- or zero-loss timely delivery, it does work very well.

Over the internet - less well, and even if your target networks are 
multicast-enabled, something like NORM or FEC is required to fill in the 
gaps, which inevitably adds latency.

Add to which the fact that you want your protocol to be 
congestion-control compatible, which is seriously challenging with real 
multicast, and if you want crypto you've got to have some kind of group 
ciphering system..

> uses for server-to-server communication.  If you have 100,000
> concurrent, active connections, you're already probably beyond the
> CPU constraints of a single machine.  So have your clients connect
> (via round-robin DNS, or whatever other mechanism makes sense) to a
> variety of different servers, then have the servers connected to each
> other in a pattern such that one server tells ten friends, and they
> each tell ten friends, and so on, until each server is only
> responsible for spewing out data to a reasonable number of
> connections (5000 maybe?).

I agree, this is probably the best model at the moment. It has the 
advantage that if you need to scale globally, you can push the "caching" 
nodes closer to the clients, and use geographical-aware routing to push 
clients onto closer nodes.

More information about the Twisted-Python mailing list