[Twisted-Python] Operating on metainformation in a distributed system

[Krysk]

Hello. I tried to ask this on StackOverflow, but Glyph advised me this
would probably be better. I've had trouble phrasing these questions
well, so please ask me if you need more information.

I'm working on a game server that needs to support 18,000+ players.
The game requires fairly intense resource use, and I'd like to have an
upper limit on the playercount much higher than I need. The solution
seems obvious: Design a server that can scale out and up as necessary.

Because distributed systems are hard, I tried to simplify the design
so that isn't a concern as much as possible. To that end, my
architecure is pretty simple. A player is always assigned to a
GameHandler instance. By default, a player is assigned to a
Lobby(GameHandler) instance. They can then queue for a match, and when
an appropriate match is found the server with the least load creates a
new handler, say, CaptureTheFlag(GameHandler). Then, the servers which
those players connected to serve as reverse proxies, forwarding all
data to the CaptureTheFlag handler. When that game ends, those players
are all returned to their Lobby(GameHandler) instances. Reverse
proxies are neccessary because I didn't write the game client and
modifying it is not an option. Connections cannot be renegotiated. I
can place all the servers in the same LAN, which should prevent any
major latency issues, and make bandwidth not a problem.

So far, all is good, I think this design will work well and be very
simple to work on. However, it raises the big, ugly question: How do I
share metadata across the distributed nodes?

That's necessary for the matchmaking itself. We might have 400 players
connected across 10 servers, and we want to make a match where there's
eight players on one, four on another, and four on another. I also
need to be able to figure out how many players are on the entire
network, syncronize bans and configuration data, etc.

I was thinking I could use MySQL to store the configuration data, and
use Redis for the transient data like who's online, who's in queue,
etc. Then I could have one server dedicated to operating on all that
data (such as arranging fair matches). I could use some kind of push
notification to let servers know when a match has started or ended, or
just have them query Redis periodically.

This doesn't seem very elegant, easy to work with, or easy to
implement, so naturally I don't like it very much. I'm sure it will
work, but I was hoping someone could suggest a more natural approach.

Thanks in advance,
Krysk