Low traffic client synchronization with server in MMO

Question

I am implementing MMO where player flies in space on his starship controlling it with arrow keys and cooperate with other players.

I want to implement it so that player will be able to dodge his ship from rocket or something else, so I am trying to predict whole game state on client side using the same world simulating algorithm as server use. That game world is written on C# and will be called within client directly (it's written on Unity3D) and through CLR on C++ server (under Linux). Connection through UDP.

Problem is how to maintain, for example, 1000 players within single map (exclusing all other game objects, mobs...): Let's say I will:

• synchronize server with clients 50 times per second
• send to each client states of just that game objects (and players) which he is able to see (within some radius)
• have to send 100 objects to each player within his view radius
• must send averagely 50 bytes per game object (it's id, x,y coords, rotation, state...)

so, it will need to have such network bandwidth: 1000 (clients) * 50 (times per second) * 100 (objects to send to each player) * 50 (bytes per object) = 250 000 000 bytes per second! It's impossible!

Is it possible to reduce this value somehow? For example let clients to fully simulate their game worlds (for a long period of time) and send them just inputs of other clients and synchronize game worlds, let's say, each several seconds but it will cause weird desynchronization issues at leash because of float computations.

Anyway, how such games are programmed in common way? Thank you.

Answer 1

You only need about 30 updates (or even less maybe 10 or 20) per second. interpolate the positions of moving objectts client sided. In general you should only send data when it's REALLY needed. In WoW you might receive more updates from the players you are in a group with than from the players that are in the same location. Also, if another player is far away from you, you don't receive as many updates per second about him.

Then, only send one complete snapshot to each player when he connects. After that only send the changes of game objects. If no change occured, don't send it.

Then, make heavy use of BitVectors or howevery you might call them to reduce the amount of unneeded data! Example: You can also try to write a float using only one byte(in a range from 0 to 1 or -1 to 1) so you only have 256 or 128 different values. But the player will not notice any jerky movements thanks to the interpolations.

Look at this for an example with LidgrenLibrary on how to compress data: http://code.google.com/p/lidgren-network-gen3/wiki/Optimization

Next: Try to reduce view radius of players as they move, and only transmit important information in that time. Then when they stop increase their view radius again. You can use a spatial hashing system or a bsp tree to reduce the overhead of looking up objects that are "in range". This is a good read for the topic: http://en.wikipedia.org/wiki/Collision_detection

Also compress the data YOURSELF only YOU know about the data structure and the temporal coherence in the data (which can and should be exploited). A general algorithm such as Bzip2, Deflate, whatever, should be used, but only as the final stage of compression!

Also, for non game-critical information, you could also employ additional P2P techniques. Example: A player plays the "hello" animation.(Just a graphical effect) The player sends this information to the server, but the server doesn't relay the information to the other players. Instead this non-critical effect is send by the player itself to the other clients in range.

EDIT (because of the comment):

Additional methods to decrease the average bit count per second to each player:

1. You wrote that you send "Object did not change". There is no reason to do this. If you worry about packet loss (and getting your simulation out of sync because of this) consider the following: At each fixed timestep (ex. 100, 200, 300, 400...) hash the simulation state and send it to the server. the server confirms or sends a complete snapshot of all data back.

2. For things like rockets or even players you can employ not only interpolation but also extrapolation in order to make the simulation more realistic. Example 'Rocket': Instead of updating with messages like "Is now at position x" just send a message once containing the following: "Rocket Spawned: position(vector), Time(at which simulation step the rocket was spawned), velocity(vector)". So you don't even have to include the rotation because the tip will always be in the "velocity" direction.

3. Combine multiple commands in one message and don't ever send messages smaller than 16-20bytes because the udp header will be bigger than the message itself. Also don't send packages bigger than the MTU of your protocol because fragmentation will slow down the speed of the transmission.

Answer 2

Here are two approaches:

First:
Switch to deterministic physics, send player commands, ai actions, objects coming into view and whatever can't be determined client-side to clients. This must include non-commands, a confirmation that up to a certain point in time nothing but the commands that has been sent and received apply.

The client must run two or three simultaneous simulations.
1: Halts whenever missing data for the next step.
2: Continue using guess data and provide the state used for rendering. 3: Whenever no 1 comes to a halt this simulation copies the state of no 1, catch up to the current time and take over for no 2, which is then dropped.

If the catch up is fast enough you can leave out differing between no 2 and no 3 and just drop the old data immediately.

Second:
Don't use deterministic physics, do the same as above, but send "full frames" once every few seconds. You can easily completely leave out transferring temporary stuff like bullets.

In both cases you might want to be wary about the client predicting anyone dying, it's kinda silly seeing an opponent unexplode.

And +1 for doing the maths, too many people fail to make simple resource use estimates.

Answer 3

Few questions first.

Are the 'rockets or something else' intelligent or dumb? If they're dumb all you need is the timestamp of fire, the origin, and the vector to simulate their path. If they're intelligent how intelligent are they? Can you compute at the time of fire that they're going to hit or miss? If so you can simulate the entire path on the client. ("At T13 the missile will strike the ship because the play lost the dodge roll / the shooter scored a critical hit.")

In general though there's pretty much no reason to: A) have a clock rate of 50Hz, (Most shooters get away with 15-20 and MMOs less than that.) B) send full state every frame. (Does the rotation of a missile in space matter at all? Or can you just assume that it's 'front' is orientated along the vector it's traveling?)

Spend time with prediction and interpolation, and you'll see your bandwidth plummet. A project I worked on had an update rate of 10Hz, and an object state representation of I think 14 bytes. (Compress everything you can! I believe we used 6 bits to define the rotation around the x plane and then another 6 bits for a tilt above/below that plane, it looked indistinguishable from sending an actual rotational matrix / quaternion.)

Another thing you can do is prioritize objects. Show, maybe there's 100 objects in the relevant set, but do you know his view frustum on the server? If something isn't in his view can you drop it's update frequency by an order of magnitude?

The general idea isn't to make a perfect simulation on the client, that's impossible, the idea is to make a fun game where the players won't notice that it's not a perfect simulation.