How do I write a game server with single-threaded networking?

Question

I want to write a game server with one thread for clients message handling and use something like epoll to accept network messages. All I/O and database access will be processed in a thread pool and they are asynchronous and non-blocking.

Given such a scheme, how should I handle other events/messages (physics, game logic, etc)? Can I implement methods to handle these messages as a blocking interface in the main thread?

What simple, popular servers exist (using Cocos*d/Unity as client) that I could look at for an example of this?

Answer 1

A general game loop should suffice, regardless of how many "games" you're handling. I would suggest considering having a threadpool for your "games" and a separate thread pool for your I/O and DB stuff. Say a 16-core server would use 8 threads per pool would be a good start.

The concept is that your main thread runs through the current number of active games and runs a frame on them. The frame does everything from collecting current network messages, handling physics, updating objects, etc.... Then once that frame is done, the main thread runs a frame on the next game instance, and just constantly loops this.

This is how a little card game server concept of mine behaves, and it seems to work just fine with ~10 active sessions, 2-8 people per session, on a 4-core 2.66ghz system that I tested it on. One thing to look out for is that if one "frame" takes an abnormal amount of time, it will severely affect the performance of the rest.

Therefore if you're going with this solution I highly recommend that everything you update is able to take variable delta time steps with potentially large steps without breaking. It means making sure that your physics updates are done correctly, and placing certain limiters so as to avoid those possible slow downs.

As far as open-source solutions, not really the best place to ask for one. Who knows if someone might have something like that, but you're probably better off researching that bit on your own.

Answer 2

In a single-threaded environment, there is usually nothing wrong with processing events right away when they happen.

But in a multi-threaded environment this can lead to race conditions and syncronization issues. For that reason I would recommend you to give each thread an event queue to which other threads can push events. Each thread would lock its event queue in a suitable phase of their loop, process the events in the order they were received, and unlock the queue again.

Make sure you use the appropriate synchronization features of your programming language to ensure that pushing on and processing of the event queues is thread-safe.

Answer 3

Although this is nothing game specific, libuv may be interesting way to implement a server. libuv is the underlying core that is powering node.js, but instead of JavaScript you program in C. (No big deal to integrate into C++.)

The key point behind libuv (and node.js for that matter) is that all the high level logic is executed in the main thread and all lowlevel logic and I/O is offloaded asynchronously.

The classical example is a database, you formulate the request to the database in the main loop, fire off the request and while you wait on the result something else is done. Once the results are back, you process the result in the main thread. Buy using the function uv_queue_work, you can offload anything on a pool of threads. The awesome thing is, if you factor your code properly you can work with no or almost no synchronization.

In the case of a game server I can imagine something along the lines of the following:

You open a UDP socket and wait for input (not blocking the main thread). When something comes in you handle the request (parsing the UDP packet into a message could be offloaded). And formulate a request against the game world, once that is done you build the answer asynchronously.

While all this is happening every 10ms you trigger a physics update which is executed async. You would sync the game state with the physics kernel in the main loop, but the actual physics computation would be done in a different thread. (All without locking.)