[Twisted-Python] pushing out same message on 100k TCPs
tobias.oberstein at tavendo.de
Sat Feb 11 07:00:42 EST 2012
> Not to mention the fact that inevitably, you probably are going to want some
> security on those connections, which means TLS, which means individual crypto
If there is a need for encryption, then yes, .. but it's not always needed
> I believe the best model for this kind of high-volume reliable-multicast-over-
> unicast is a spanning tree, like what IRC uses for server-to-server
Ok. Makes sense. I see .. could have frontend nodes, which act as aggregators
condensing the messages on many incoming connections into 1 uplink TCP
connection to a backend node .. or a hierarchy.
This is quite interesting .. for I am now beginning to see why Google is
pushing the discussion on WebSocket multiplexing extension on Hybi;)
Until now, I thought that would be primarily because Chrome has a
process per tab design .. but that's probably only one reaon. The cloud
side being the other one ..
> communication. If you have 100,000 concurrent, active connections, you're
> already probably beyond the CPU constraints of a single machine. So have your
Not really. I've been testing up to 180k connections on a 2 core virtual machine
and when doing PubSub over those (dispatching 1 msg to all 180k) .. CPU load
isn't maxed out .. but I'm hitting a limit of around 20k/s dispatched msgs.
My suspicion is it's probably more interrupt/syscall overhead bound ..
> clients connect (via round-robin DNS, or whatever other mechanism makes
> sense) to a variety of different servers, then have the servers connected to each
> other in a pattern such that one server tells ten friends, and they each tell ten
> friends, and so on, until each server is only responsible for spewing out data to
> a reasonable number of connections (5000 maybe?).
5k is much too low .. would require too many servers. My target is 200k per node
and have as high as possible msgs/s dispatch rate.
Probably a design with frontend nodes aggregating TCP connections is inevitable ..
More information about the Twisted-Python