game-state update structure and distribution in MMO?

Question

I'm developing an MMO, which has to maintain a global game-state.

Now I want to distribute the updates, which the players make to this state, to players which can see those updates (area of interest etc.)

Is there a common structure/format for these updates and for a global game-state?

Like, every action a player sends to the server has to include a type, the coordinates etc.

And are the updates, which gets distributed from the server to the players in the same format or are they different?

Also should I include timestamps or use the receive-time from the server to check which action came first and which sencond etc.?

And what should I send back to the player if he send an invalid action?

Answer 1

I recommend checking out already existing documentation on network protocol of games like Unreal, Quake, Tribes. For example here is a document for tribes:

http://www.pingz.com/wordpress/wp-content/uploads/2009/11/tribes_networking_model.pdf

You may want to separate your updates into the following categories :

reliable - things that the player has to know about unreliable - things that the player should know about, but no big deal if it is missed from time to time local - things that do not impact other players

For reliable messages (you can send multiple messages in a batch as well), you will most likely need some sequence numbers so that all updates are applied in chronological order and that both client and server are always kept in sync. If the player is trying to loot something, then the player would send a reliable message, if the server does not ack the reliable message (packet loss) then player has to resend the request (or timeout). Neither the player, not the server change their state until ack is received (for reliable messages).

Unreliable messages are things that can be missed once in a while, like players movement, the direction where the player is looking, the appearance etc. If the player loses these packets, at the next update, this information may be resent later, no big deal. If no ack is received, then the server will eventuall resend the unreliable update, if ack is received for unreliable message at some point then the server will stop sending the unreliable updates. Also if the player changes position, there is no need to resend the old position, just resend the latest state until player acks it.

Local updates are things that each player may see differently but have no impact on the game, like animation of trees, weather effects, things for which a global state does not need to be kept.

Some other things worth considering :

Do not flood the network:

You do not want to send unreliable updates every time a player moves (ie packet per move), that will just flood the network, you want to send them at a certain interval (like having fps and sending the update every 10-15 frames). Also if you send every unreliable event, you will slow down the performance of your server, so you may want to filter out how many unreliable events you send. If there are 500 players in the room, moving around, you may not want to increase the refresh interval and or prioritize which state updates are more important.

Path prediction:

The client should have a prediction built in for the movement so at frame 0 you can say the player is moving towards this coordinate and at frame 10 you can say that the player has changed position and direction, the client has to interpolate what happens between frame 0 and frame 10.

Timestamp:

Ideally you want to use sequence numbers for tracking game state, every time you send any messages there should be a sequence number (for resending purposes), reliable messages should have another sequence number on top of your packet sequence numbers. You can also include timestamp (simulation time) every few packets.

Protocol/structures:

You want to pack your data as much as possible (so that you can send more information in less time). Use tricks like bitvectors to store flags. Instead of strings send string ids (unless its chat). You will most likely have large switch statements for encoding/decoding, you may want to build a tool to autogenerate these or use something like google protobuf.

Error handling:

If the player sends a reliable request, then the player can either get a valid reply or an error reply (or timeout and disconnect), for example if the player wants to purchase an item, he can either get a reply saying item is purchased or an error that the item is not available or has already been bought by somebody else, until any of the possible responses is received, the player should do nothing (no state update).

Answer 2

The basic networking strategy looks like follows:

Every action the player does should be sent to the server with as much data as needed for the functioning of the game, but not more, to ease on the network traffic. Every data needs to be verified by the server to be a valid action. Invalid actions are logged for review, since they indicate bugs or cheating attempts.

The one overarching rule is to never trust the client with anything of importance: The client is in the hands of the enemy. This includes the time stamp, where the server should always dictate it (and thus the order of actions). What the packets should have is a serial number, so that the server can detect network problems easier and so that actions of a client happen in the same order as the client requested it. Two packets with the same serial number but different content also indicate a hacking or cheating attempt and should be marked as invalid and logged as above.

The state updates on the server have to happen atomically, though in practice this requirement is hard to fulfil without severe performance penalties, so most games opt for "locally atomic" instead (updates which happen "far away" and can't influence the state of the actor don't need to by synchronised with actions of the actor). This is a consistency/performance trade-off. Problems related to it have been a source of exploits like duplication bugs even in major games in the past.

The server then gathers the updates for each client into a per-client queue (usually the size of your networking data window, so something on the order of 1.5kb) and sends them after either a specified timeout or when the queue is full. The updates are bundled together in one packet - your system needs to be aware of it and handle multi-update packets. The server never sends client packets unfiltered to a client, not even to the same client. In particular, any text has to be filtered to include only the "safe" character sequences and any updates which the other client shouldn't be aware of aren't sent at all. Any actions which failed, for whatever reason including an invalid action, get a "failed action with serial number XXXX" notification sent along with the updates to the client which originated it.