How do I smooth out latency jitter in my netcode?

Question

I'm developing a multiplayer game and I'm following some articles I've found online to create an authoritative client-server model.

Reading this article, Valve says that every time a message is received, the client assumes the "current" time is the time in the latest packet, in the article they state it:

It is assumed in this paper that the client clock is directly synchronized to the server clock modulo the latency of the connection. In other words, the server sends the client, in each update, the value of the server's clock and the client adopts that value as its clock. Thus, the server and client clocks will always be matched, with the client running the same timing somewhat in the past (the amount in the past is equal to the client's current latency). Smoothing out discrepancies in the client clock can be solved in various ways.

And interpolation of remote entities:

1. Each update contains the server time stamp for when it was generated
2. From the current client time, the client computes a target time by subtracting the interpolation time delta (100 ms)
3. If the target time is in between the timestamp of the last update and the one before that, then those timestamps determine what fraction of the time gap has passed.
4. This fraction is used to interpolate any values (e.g., position and angles).

Now obviously due to jitter, my interpolation looks terrible as I will receive time + latency jitter. So my question is, is on this part: "Smoothing out discrepancies in the client clock can be solved in various ways."

What would be a good way to smooth out any discrepancies?

Answer 1

Update: so this is the approach we decided to go for, both for the sake of testing our solution, and if it being a valid solution for future use of anyone else who might need it.

So instead of directly setting the time on the client when a remote packet is received:

1. we store the "debt time" between the client predicted time and the received time
2. If said debt time is higher than a certain treshold, we snap the time to the servers time and reset the debt (causes snap) and stop here
3. If not, then we smooth it out throughout several frames (by max 10% of a frame duration). This causes the entities to slow down and speed up on jitter, but it's very unnoticeable and gives us good results

Answer 2

In my implementation simulation on both client and server is running in discrete "ticks", so I can always slow down client, so it is few ticks behind the currently arriving data according to jitter average, so unless there is "lag spike", the simulation is smooth.

Answer 3

Gaffer on Games, an experienced authority on online multiplayer games, recommends having the clients show one or two ticks behind the server's state.

When the client receives state 100, for example, it starts interpolating from the state it already has (state 99) and state 100 such that it should reach state 100 at the same time that it receives state 101 from the server. The server sends state updates at a fixed rate, so this is predictable, but it does inherently introduce a small bit of extra latency (based on the update rate).

If you want to cover up the occasional lost packet, have the client show two states behind the server. For example, when it receives state 100 it starts interpolating from state 98 to 99, such that it should be showing state 99 when state 101 is received. If state 101 doesn't come through (due to a lost packet), it continues interpolating between 99 and 100, anticipating 102 to come through next. When state 102 comes through, it can interpolate between states 100 and 102 over two update cycles, and the player should be none the wiser. This, of course, doubles that extra latency, but with a high enough update rate, is not a problem.

---

Edit to better answer your question according to your comment. I've kept the previous part as it informs the example in the latter part:

Where you have network jitter, my suggestion is that when updates come later than expected (or two in a row do, if you use the two-cycle smoothing described above), they set the new expectation for latency. When packets come sooner than expected, instead of jumping ahead in time, store those states in a buffer and continue catching up at a consistent rate, assuming that packets may still come through with as much latency as your worst case thus far. Of course, you wouldn't want an unlucky spike in latency to add artificial lag to the rest of the session, so as packets continue to come in sooner than expected, slowly bring that simulation lag down to catch up with the server, perhaps with a buffer if the incoming states vary quite a bit in timing.

As an example, you might expect an update rate of 20 updates a second. That's an update every 50ms. If update 100 comes through slower than expected and you aren't using an extra update cycle of smoothing, you're forced to wait for it. But once it does come through, you interpolate between state 99 and 100 such that you should reach state 100 in 50ms. In that time, since you have all that network jitter, state 101 and 102 arrive. But instead of jumping ahead to them, you store them, consistently taking 50ms to move from one state to another. As you continue to sit a few states behind what you've received, you decide to play catch up and start moving from one state to another in only 49ms (or even less), subtly bringing the player forward in time to enjoy the improved latency their connection has been having without any abrupt changes in simulation speed.

In anticipation of further inconsistencies, you may want to remember the worst delay you've had in the last x number of seconds and stop catching up when you reach that delay again (that is, return to 50ms between each state of you're expecting 20 updates per second).

With a two cycle delay, you can ignore one-off delays, and need only remember the best of two consecutive delayed states in the last x seconds. It could be worth switching between a single cycle delay and two cycle delay depending on the quality of the connection.

I hope that's helpful, and if not, hopefully someone with experience in that area can correct me and teach us both!