Is it feasible to use a DHT to span an MMO over many servers for improving reliability?

Question

At the moment I have an idea for how to handle distributing MMO servers. At the core is a bunch of worker nodes that all share the same set of distributed hash tables. Each DHT stores a specific game component. Each frame a coordinator server (which there can also be many of) pushes work out to all of the connected worker nodes and wait for them to complete all of the assigned tasks.

The system is also divided up into four parts: the client, which obviously the player uses. The proxy server that sits behind load balancers and users connect to. All they do is process incoming user packets, verify them and passes the request to the work cluster. The work cluster receives packets from proxies and append them to a process queue and wait for the frame event from the controller node to process them.

I just want to know if adding a DHT to the worker nodes like this would be feasible? This should allow redundency and high availability out of the box as DHT's are quite good at that.

Answer 1

I think the overhead of using DHT is too big. Normally a component is looked up in a local hash table. If you're lucky that is a single hash operation and a memory look-up. These operations are in the order of nanoseconds

Using distributed hash tables means that two servers would need to talk together. Using a normal local networking connection this is in the order of milliseconds, or a million times slower.

Even if you would use something as outlandish as Infiniband to connect the two servers the latency would still be in the order of microseconds, or a thousand times slower.

And this is just the connection latency we are talking about. There is also the cost of finding the data on the server and then sending it. All of which takes a lot more time than computing that single pointer to memory on a single machine.

Answer 2

I would say, as somebody who has worked on MMOs, that this approach seems a little half-baked (perhaps that's just because you're omitting portions of it?), but more importantly takes far, far too granular approach to the distribution of tasks across a network of machines.

The overhead of the intercommunication protocol would impact your performance significantly, and debugging the thing would be quite a nightmare.

A far better approach would be to distribute work at a higher level, choosing a level of granularity that maintains as much local coherency as possible so all your processing for "nearby" actions, players, events, et cetera occurs on one server (on Guild Wars 2, we distributed work at the map / region level, for example -- although note that this is not how the "sharding" in GW2 worked, that was a done differently and for a different reason).