11
\$\begingroup\$

Web/Mobile full stack engineer here. Zero game experience.

How do big multiplayer games handle multiple database updates simultaneously?

I.e. World of Warcraft or similar games with 10k players playing at the same time say defeat 10-20 monsters at once, it drops items & the player gets experience. This is probably 5-15 database updates per player simultaneously

Please let me know if the question isn’t clear or needs further clarification

\$\endgroup\$
6
  • 7
    \$\begingroup\$ Is there a reason why 300 database updates simultaneously should be a problem - as long as they are done asynchronously? How many operations per second can Redis do? Google says about 300,000 updates per second. On one server. \$\endgroup\$
    – user253751
    Aug 31, 2022 at 17:00
  • 2
    \$\begingroup\$ See also How does mmorpg store data?. \$\endgroup\$
    – Theraot
    Sep 2, 2022 at 18:02
  • 1
    \$\begingroup\$ By running more than one server and denying new players joining it until they just decide to sever the connection and make you have update if you try to join a new one. - database updates is different. How do online games' [databases] handle frequent updates? \$\endgroup\$
    – Mazura
    Sep 2, 2022 at 23:29
  • 2
    \$\begingroup\$ Web servers are stateless. And thus for each request they have to load said state. Game servers are [usually] stateful. They can get away with keeping stuff in memory. \$\endgroup\$
    – n0rd
    Sep 3, 2022 at 2:41
  • 2
    \$\begingroup\$ "I.e. World of Warcraft or similar games with 10k players playing at the same time" Not all 10k are actually doing much at the same time and they aren't all on the same server either. Those two things help a lot. \$\endgroup\$
    – Mast
    Sep 3, 2022 at 7:51

4 Answers 4

35
\$\begingroup\$

While web applications tend to store everything in the database immediately, games work a bit differently.

Any information pertaining to the players currently in the game is stored in the RAM of the gameserver. This is not just to keep load off of the backend database, but also for performance reasons. Games require very low latency and do a lot of number crunching. Delegating all that to the database would result in unacceptable response times.

And then there is a lot of information which really does not need persisting. Like the positions of monsters, for example. When a server reboots, then nobody will care that the monsters don't spawn in the exact same locations as they were when the server went down.

The information the game does need to persist, mostly the state of the player-characters, usually does not get persisted to the backend database in real-time. Usually it only gets persisted when the client disconnects, the server goes offline or at regular intervals (to not lose too much game-state in case of a server crash).

And because character-states are usually persisted and restored in an all-or-nothing manner, I would argue that it is questionable that there is any benefit to properly normalize all the data of the character gamestate. So instead of making different database tables for character stats, character inventories, character quest states etc. it can make sense to just store all the information about each character as a BLOB. The drawback is that BLOBs are far harder to analyze and data-mine. But you can solve that by importing all that data into a properly normalized secondary database used only for analytics. That data doesn't need to be perfectly up-to-date, so you can do that as a daily job which runs during the time where you have the least players online. And then the analytics team can do whatever they want with their analytics database without affecting the actual game.

\$\endgroup\$
8
  • 2
    \$\begingroup\$ It should also be emphasized that there is significantly less distance between "in RAM" and "in a database" than one might expect, assuming you consider eventually-consistent NoSQL solutions to constitute a "database." Many of those systems are basically RAM with asynchronous persistent backing. \$\endgroup\$
    – Kevin
    Aug 31, 2022 at 20:38
  • 4
    \$\begingroup\$ @Kevin Even the fastest NoSQL in-memory database is still a separate process with which you communicate via a network protocol. That's still ages compared to just accessing a memory address. \$\endgroup\$
    – Philipp
    Aug 31, 2022 at 22:27
  • 2
    \$\begingroup\$ @Philipp: The user is communicating with the game server via a network protocol, over an actual network. You can at least put the database on the same box as the server. But I agree that, for maximum throughput, you do it in-memory first and then asynchronously tell the database about it. I just don't think the game server ought to be bothering with anything so complicated as batching up writes when the database can already do that perfectly well on its own. \$\endgroup\$
    – Kevin
    Aug 31, 2022 at 22:33
  • 3
    \$\begingroup\$ @user253751 A number of games do, in fact, have forced sync points for certain events. For example, trading items between characters is often a trigger for a forced sync of the character state to persistent storage for both characters (because this prevents a whole class of potentially game-breaking bugs). \$\endgroup\$ Sep 1, 2022 at 1:34
  • 3
    \$\begingroup\$ And don't forget about data stored and calculated client-side, which (in developers' opinion) were too irrelevant to be calculated server-side. Until the players discover and exploit that. A common example: character location (so every step is not sent to the server), leading to various kinds of speed and teleport hacks. \$\endgroup\$
    – Trang Oul
    Sep 1, 2022 at 6:20
6
\$\begingroup\$

Firstly, as a comment points out, a properly configured database can be pretty fast, especially in 2022. But MMOs have been around for thirty years, so it's worth looking at the techniques rather than relying on Moore's law to save us.

I.e. World of Warcraft or similar games with 10k players playing at the same time say defeat 10-20 monsters at once, it drops items & the player gets experience. This is probably 5-15 database updates per player simultaneously

The big trick is that they're not all playing at the same time in the same place. WoW segments its players by server, and then further groups them by "instance", so that there's only ever a much smaller number directly interacting. That allows rapid updates to be made to the instance's copy of the data, which are then persisted to permanent storage at a lower rate.

There are very few MMOs which allow you to have everyone in the same place at the same time. The big exception is EVE Online, which groups players by star system but can still support battles with thousands at one time.

\$\endgroup\$
4
\$\begingroup\$

When i was working on a MMORPG (2006-2011ish), we loaded character data from MySQL on login, and held all updates in RAM, until character logout/avatar death, upon which the MySQL database was updated and the RAM released. Also each login was a dedicated OS-level thread, and each player-thread spent the majority of their time synchronizing with the main "game thread" (for example, a player sent a packet for "throw my weapon on the floor", the player thread receives it, tells the main game thread, then the main game thread updates the inventory and floor tile in-ram, and tells all player threads whose avatar is in the vicinity, "update the item list on this floor tile"~)

  • the SQL queries was (mostly) only on login and logout, and ran in their own dedicated thread.
  • the server had a max limit of 666 concurrent logged in players, and that limit was rarely reached (right now its 347 players, but i no longer work on it, and i don't know if that limit still exist)
\$\endgroup\$
5
  • 1
    \$\begingroup\$ Sounds dangerous for when a server crashes. All player progress is lost. \$\endgroup\$
    – Vaillancourt
    Sep 2, 2022 at 18:46
  • 4
    \$\begingroup\$ @Vaillancourt true, but it was rarely an issue. There was at times, players who figured out some way to crash the server, and duplicate rare/expensive items with it, like "2 players login, player1 gives rare item to player2, player2 logout, player1 crash the server", and after server restart, both player1 and player2 have rare item. luckily we had crash dumps and packet logging for that.. it happened like.. maybe 3-4 times in 5 years \$\endgroup\$
    – hanshenrik
    Sep 2, 2022 at 18:53
  • 2
    \$\begingroup\$ Yes, I understand the technical trade-offs there. This story reminds me of a clan in an MMO that never lost a battle while protecting their fortress because they managed to crash the server by starting to dance all at once in front of it. I can't remember which MMO this was, though. \$\endgroup\$
    – Vaillancourt
    Sep 2, 2022 at 18:56
  • \$\begingroup\$ @Vaillancourt haha neat, do you know the server language? was it C++? (mine was C++ and LuaJIT) \$\endgroup\$
    – hanshenrik
    Sep 2, 2022 at 19:21
  • 2
    \$\begingroup\$ No, I don't recall. It was not an issue with the server language; as I recall, the issue was caused because this created a N*N set of messages between players "hey, I'm dancing" "Hey, look everyone! PlayerX is dancing". I don't remember the details. \$\endgroup\$
    – Vaillancourt
    Sep 2, 2022 at 19:24
0
\$\begingroup\$

When I worked on MMORPGs main issue for such engines was RAM for database handling.

All stuff is in database and it's just changes of database values upon actions on client. Then depending on game code quality there's anticheat which detects separately whether the client is trying to game speed in database (that's why there is a lot of cheaters in online games).

So say you are slashing a dude in MMORPG - that means your client sends to server attack command and receives health and modified values. If you use potion it is sent in a separate command so sometimes lag or network drops can destroy your winnings or potion use or something because that particular packet didn't get through and invalidated request.

For speed most of such exchanges are not really protected so it allows a lot of cheating to happen.

Most of games are made with client doing most of the work because most game programmers have no understanding of proper network programming (with multithreading and asynchronous io. most have no idea about this.)

\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .