1
\$\begingroup\$

At the moment I have an idea for how to handle distributing MMO servers. At the core is a bunch of worker nodes that all share the same set of distributed hash tables. Each DHT stores a specific game component. Each frame a coordinator server (which there can also be many of) pushes work out to all of the connected worker nodes and wait for them to complete all of the assigned tasks.

The system is also divided up into four parts: the client, which obviously the player uses. The proxy server that sits behind load balancers and users connect to. All they do is process incoming user packets, verify them and passes the request to the work cluster. The work cluster receives packets from proxies and append them to a process queue and wait for the frame event from the controller node to process them.

I just want to know if adding a DHT to the worker nodes like this would be feasible? This should allow redundency and high availability out of the box as DHT's are quite good at that.

\$\endgroup\$

2 Answers 2

2
\$\begingroup\$

I think the overhead of using DHT is too big. Normally a component is looked up in a local hash table. If you're lucky that is a single hash operation and a memory look-up. These operations are in the order of nanoseconds

Using distributed hash tables means that two servers would need to talk together. Using a normal local networking connection this is in the order of milliseconds, or a million times slower.

Even if you would use something as outlandish as Infiniband to connect the two servers the latency would still be in the order of microseconds, or a thousand times slower.

And this is just the connection latency we are talking about. There is also the cost of finding the data on the server and then sending it. All of which takes a lot more time than computing that single pointer to memory on a single machine.

\$\endgroup\$
6
  • \$\begingroup\$ Yes yes I do understand that, it will be much slower than a local memory lookup. Hmmm looks like I have to do a bit more research and benchmarks. What other methods can maybe be used to distribute the load across different servers? \$\endgroup\$ Feb 26, 2014 at 5:01
  • \$\begingroup\$ Well sharding seems to be the most common approach. Check out how games like Eve-Online do this. \$\endgroup\$
    – Roy T.
    Feb 26, 2014 at 10:51
  • \$\begingroup\$ @MrWiggels You could divide your game world into zones and calculate each zone on a different server. \$\endgroup\$
    – Philipp
    Feb 26, 2014 at 12:11
  • \$\begingroup\$ (Ahum, that is exactly what sharding is ;) ) \$\endgroup\$
    – Roy T.
    Feb 26, 2014 at 14:10
  • \$\begingroup\$ @RoyT. A Shard in the context of an MMORPGs usually means distributing the player population on multiple servers where each server is simulating a whole, independent world. \$\endgroup\$
    – Philipp
    Feb 26, 2014 at 21:15
2
\$\begingroup\$

I would say, as somebody who has worked on MMOs, that this approach seems a little half-baked (perhaps that's just because you're omitting portions of it?), but more importantly takes far, far too granular approach to the distribution of tasks across a network of machines.

The overhead of the intercommunication protocol would impact your performance significantly, and debugging the thing would be quite a nightmare.

A far better approach would be to distribute work at a higher level, choosing a level of granularity that maintains as much local coherency as possible so all your processing for "nearby" actions, players, events, et cetera occurs on one server (on Guild Wars 2, we distributed work at the map / region level, for example -- although note that this is not how the "sharding" in GW2 worked, that was a done differently and for a different reason).

\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .