Multithreaded design for a game server

Question

I would like to start developing my own game. Basically, the game has a "world" that players can travel around and modify. Think of it as a much more realistic version of Minecraft or an open world MMO.

I have thought of two possible ways of designing the server, both with problems.

The first is a single threaded design where one thread handles all the game mechanics and player interactions. The problem with this is that it cannot fully utilize a processor, and if one player does actions that cause lag, all players will experience that lag.

My next idea was that there is a separate thread for each player that renders the terrain around the player, handles physics, NPC interaction, etc. With this approach the CPU could be fully utilized and one player causing lag wouldn't affect others. However, the problem is keeping the different threads in sync. The entire server must be on the same "clock," (same day/night cycle, certain things take a certain amount of time, etc.), i.e., the different threads must be synchronized. However, this raises the problem that if one player's thread is lagging behind, the only way to synchronize the other threads would be to also cause them to lag, and this would eliminate one of the big advantages of a multithreaded game server, not to mention that keeping everything in sync can become extremely complicated.

So my question is, what would be the best way to design the game server using a multithreaded design where the server is kept in sync and one player causing lag won't affect others? Or is this even possible without dropping a significant amount of game ticks?

Answer 1

The greatest problem when it comes to parallelization is sharing data between threads. When multiple threads read and write the same data structure simultaneously, then you will have a myriad of problems. Either you get very difficult to reproduce bugs due to race conditions (bugs which only happen when one thread writes something while another thread reads that information) or your threads end up wasting a lot of CPU time with waiting at thread synchronization points.

So when you are designing multithreaded architectures, then what you should look for are units of data processing which require little to no communication with each other.

"One thread per player" is usually a bad choice as a unit of parallelization. All the players interact with the world and with each others all the time. There is not much you can parallelize here in good conscience.

Dividing the world into cells

A common approach in an MMO is to spatially divide the game world into cells and process each cell of the world in a separate thread. This works well in games where locations can be cleanly divided, for example if you have a concept of separated "maps" or "instances". Each player only needs to know what happens in their current cell, so there is little communication between cells anyway. When a player does something that causes a spike in server load, then it will only affect players in the same cell. That also makes it easier for players and developers alike to realize the cause-effect relationship between another player doing something unusual and the game lagging in response to it.

However, this gets tricky when there is interaction between cells. When you want a seamless open world, then you will have situations where an entity in cell A interacts with an entity in cell B across a cell boundary. Now this architecture doesn't work so well anymore.

Further, players in MMOs often have a tendency to cluster in certain locations. Parallelizing by cell does nothing when the situations which really push your server to its limits are events where all players gather in the same cell.

Entity - Component - System

Another approach is to use the Entity - Component - System architecture. This style of game programming has the benefit that it is pretty easy to find units of data processing which can be parallelized. You know which system operates on which components, so you can easily spot systems which have no communication with each other and thus can be run in parallel. You also recognize pretty easily when a system processes each entity in isolation, so the system itself can be parallelized (each thread processes a subset of entities).

Some of the more advanced ECS architectures make the decision what to process on which core at runtime. They usually have a concept of "jobs" which have dependencies with other jobs. An automatic job scheduler assigns jobs to threads at runtime.