# How do I handle packet loss in a client-server network model?

In a client-server network model, the clients only send commands to the server (i.e. coordinates of a click, fire gun, etc.) and the server then runs those commands to produce a game state.

But what if the client sends a command like "fire gun" in one packet which is dropped. Does the user experience that he tapped his fire-button with nothing happening or is there some intelligent way to make sure that such commands are received by the server regardless of packet loss?

I.e continuing to send the old "fire gun" command multiple times to ensure that it at least eventually fires (tagging the command so that the server understands that it is the same command sent multiple times, not a command that is fired over and over).

I've been looking at documentation regarding Quake3, Unreal and Source Network Models but I cannot find anything on this particular problem. I do believe they have solved it though because I never fired my AWP without it actually shooting. If there was lag involved, then it just took a short while before it shot, but it's never lost the command I think.

EDIT: I've found a resource that explains one way to handle this here: http://gafferongames.com/networking-for-game-programmers/reliability-and-flow-control/

They solve it by adding a kind of loose ACK on top of UDP. I feel like there should be a smarter way to go about it however. This way of handling it is ok for the server I guess which mostly needs to make sure that it has a live connection to the and to handle things such as chat messages (which are not really time sensitive but cannot be allowed to get lost) but for the client I believe an ACK creates too much delay when a packet is lost (a full round time latency at the very least).

• I believe the correct answer is "Don't". The reason why UDP is good for something like CS/LOL/ETC... is because you are constantly sending state updates which override the previous state. Very similar to a movie stream. Its a trade off of latency vs guarantee that the server state is delivered perfectly and minimal latency seems to be the priority for you. Aug 27, 2014 at 21:13
• @ClassicThunder For the server->client communication UDP works wonders as it is because of the nature of game states as you describe. However, the client does not send game states to the server, it sends commands. These are different in nature as they do not override the previous command but instead builds upon it additively. Therefore packet loss is a great concern in client->server communication. Aug 27, 2014 at 21:23
• A loose ACK via UDP is still used without much change because it works and is time proven. You're asking how to ensure the server receives a message from the client... without ever communicating that receipt back to the client. If you've used UDP w/ ACK or TCP and found it to be unsatisfactory, under what conditions does it fail to meet your needs? I'm all for academic investigation, but this seems like a case of premature optimization. Sep 12, 2014 at 19:18

It always frustrates me how many people oversimplify the differences between TCP and UDP as "TCP is reliable but slow, while UDP is fast but unreliable".

This is not what TCP and UDP are about. TCP and UDP are two different abstractions over IP, and are used for different things. Both very good in their own fields.

TCP is a stream oriented protocol. You use TCP when you want to transfer a lot of sequential data from one point to another, maximizing the throughput (that is, minimize the amount of time it takes you to send -all- of the data). The automatic resending of lost datagrams and reordering are one of the features of TCP, but TCP also features stateful connections, flow control and congestion control. These features make TCP a really good tool for transferring large amounts of data from one point to another. Protocols such as HTTP and FTP where you want to send large chunks of data as fast as possible are meant to use TCP.

UDP, on the other hand, is a message oriented protocol. You use UDP when you want to send messages from one point to one or many points, minimizing the latency (that is, the amount of time it takes between a message is sent until it is received). In order to minimize latency, UDP does not implement datagram retransmissions or reordering. However, if you want your message-based application to support retransmissions and/or reordering, you can implement that yourself, and it's actually quite simple! Additionally, there is a bunch of fun things you can do in UDP that you simply can't do with TCP, like multicast and UDP hole punching.

TCP and UDP are both awesome technologies meant for different things, so what you want to do is to ask yourself whether the communications in your program are better represented as a stream of bytes, or as a series of messages, and a good rule of thumb is:

• If your program is about sending a stream of bytes, use TCP.

This is what the decision between TCP and UDP should be like, not about reliability, which is just one of the many differences between them, and in my opinion the least important of them all.

Just like it is possible to make a stream abstraction over UDP, it is possible (and sadly very common) to make a message abstraction over TCP.

If you do choose to make a message abstraction over TCP, remember that you're going to have to use TCP in a way it was not meant to, which means you're going to encounter some difficulties. In particular:

• Message boundaries: TCP treats data as a very long stream of bytes, and as such, has no concept of packets or messages. If you want to send messages over TCP, you're going to have to create some kind of message header that includes the length of each message.

• Buffering on send: TCP expects you to be able to produce data as fast as you can. TCP datagrams can be very large, and if the network conditions are good, a TCP stack will buffer your bytes until it can send a very large datagram therefore maximizing throughput. This is a good thing for streams, but not so much for messages. Once too many times people end up flushing the buffer on each send, or using options such as TCP_NODELAY, and even then you have no guarantees. To quote the late Richard Stevens:

2.11. How can I force a socket to send the data in its buffer?

You can't force it. Period. TCP makes up its own mind as to when it can send data. Now, normally when you call write() on a TCP socket, TCP will indeed send a segment, but there's no guarantee and no way to force this. There are lots of reasons why TCP will not send a segment: a closed window and the Nagle algorithm are two things to come immediately to mind.

• Buffering on receive: Even if you manage to flush the buffer on each send, there is no guarantee that you will get one reception for each flush you send. It is perfectly possible for you to get several messages on a single recv call, because they were buffered at the reception end (especially on networks with high packet loss rates). You then have to implement a message reception queue, which is a pretty stupid thing to do when you're using TCP.

• Heartbeats and connection drop detection: TCP has its own mechanism for detecting dropped connections, but they are usually too slow for message passing applications (the non-configurable keep alive datagram is sent about once every two hours). Once too many times people end up implementing their own hearbeat protocols inside the very same TCP stream, and then they end up asking "how to force close TIME_WAIT", "How to detect a closed connection" or something like that.

None of these problems exist if you make your message oriented program with UDP:

• Message boundaries: Each message is a separate UDP datagram.

• Buffering on send: Doesn't exist. A sendto call immediately sends a datagram.

• Buffering on receive: Doesn't exist. You will only get one datagram per recvfrom call.

• Connection drop detection: UDP is not connection oriented. You can implement a simple timeout with your own parameters to consider a client is no longer there.

If you want to use TCP for message passing because you think it's simple, don't.

Now, for the dreaded reliability "problem", you can implement simple reliability by including two fields on each message: the first one is a permanently increasing message ID, and the second one is the largest message ID you have received from a particular client. If your internal count differs too much from what the client says he has received, you just resend starting from the first message the client hasn't received. With wrapover, you can do this with just two additional bytes per message.

The best thing about your own reliability layer is that you can tune it for whatever your needs are: you may have some message types with reliability in them, while others without it. You can have various channels, each with its own sequence numbers, you can use those numbers to predict lag, and lots of other awesome stuff.

Don't run away from UDP because other people have told you "it's hard". They probably have never (seriously) used UDP, and probably have not even (seriously) used TCP. Actually UDP is much more simple and easier to understand than TCP.

• I've picked UDP because of the possibility of unreliable messages and because they are messages. I believe WebRTC (which is what I'll use) implements flow control so I won't have to put a lot of logic on top of my network implementation other than game network model specifics. Sep 17, 2014 at 9:44
• While this post contains a lot of internal information about protocols. I just cannot agree with advising new programmers to re-implement TCP mechanisms using UDP as base. This advise is same like you advised new programmers to use pure C instead of java as "you dont have to worry about classes" and "it is faster because it has no garbage collection". Sep 17, 2014 at 15:15
• @wondra: So you're just saying that people should reimplement UDP features such as message boundaries, and completely misuse TCP features such as buffering, just because of one TCP feature that you're interested in? Comparing TCP to UDP as you compare Java to C is incorrect: both TCP and UDP are high level transport layer protocols. If a comparison is to be made it would be between Java and Scheme: you can certainly do something that resembles functional programming in Java, as well as something that resembles OOP in Scheme, but there are better tools to do what you want to do. Sep 17, 2014 at 15:31
• @wondra: Also, there is nothing "internal" or low level about what I wrote. Being stream-oriented is the main feature of TCP, and being message-oriented is the main feature of UDP. This is no obscure information. Sep 17, 2014 at 15:35
• The basic interface for both TCP and UDP is pretty much the same in many languages. What is not in interface is internal. I am just saying you will never convince me to implement lots of lines for error checking (ack,numbering and CRC). Why implement garbage collection to ANSI C for a little more performance(=better optimized for the single program) over java? This is the question everybody has to answer for themselves. I say no to premature optimization (at least for beginners). If he has to ask how to do it, he is a beginner. Sep 17, 2014 at 15:42

To handle packet losses, you first need to decide which network protocol you will use: TCP or UDP.

• TCP: is reliable, but slow to send simple messages (has higher latency)

• UDP: has low latency, but to achieve your objective of not letting the user notice some packages losses, you will need to develop an extra layer that will slow your communication.

This extra layer is like the difference between UDP x TCP.

You will need to study the details of the TCP protocol and implement many features. You will need at least to develop ACK packets and packet ordering. If your layer is not well developed, it can be even slower than TCP.

Maybe you can mix both protocols in the same game.

UDP should only be used when you are delivering data that can be lost without problems. For example, if you miss the packet "enemy 1 is on field A", it is not a big problem if you repeat the enemy position from time to time. If you get the second packet with "enemy 1 is on field A" or "enemy 1 is on field B", it does not matter where it was before. You will just place the enemy in the last received field.

Now, if you don't want your player to lose ANY shots, use TCP (or a reliable modified UDP message).

• But if I don't want to use ACK? I just want to build in some redundancy so that commands are not missed unless the player is lagging badly. Or could it be that the client-server network model often uses TCP for the client->server communication of commands while using UDP for the server->client communication of full game states (which can be lost without issue)? Aug 28, 2014 at 0:54
• You can avoid using an ACK algorithm (or TCP) and develop your own algorithm for redundancy, but I would not suggest that. Is the same situation if someone decide to create its own cryptographic algorithm instead of using I widely used one like RSA. Aug 28, 2014 at 1:27
• My suggestion is to mix TCP (for information that you can't miss) with UDP (for information that will not cause problems if you miss). Aug 28, 2014 at 1:28
• That is a good suggeston I think but the reason I think there should be a better way with UDP is because information will cause TOO much problems if you miss enough packets which means that I only need to have some way of protection against small packet loss that doesn't have to work for large packet losses because then the game needs to reset itself anyway in that scenario. Aug 28, 2014 at 10:13
• I've also read that TCP induces packet loss in UDP when used in combination. Plus the players commands need to reach the server as quick as possible since it determines how quick the game feels. Aug 28, 2014 at 16:16

Unless you are interested in the details of implementing your own reliability layer using UDP, I would recommend using RakNet, especially since it has been made open source. This would be the way to go if you are focused on making a game rather than a networking protocol.

With RakNet, you can send packets with certain reliability flags such as RELIABLE_ORDERED. This ensures each packet will arrive (it is re-sent if it's not acked) and it uses an underlying ordering system so that the packets are processed in order on the receiving end. This would be very important if you are sending inputs for something like a fighting game where order is crucial for calculating moves and such. This is all very easy to accomplish using a library such as RakNet.

This will however increase the bandwidth, but you can choose appropriately which packets need to be reliable and ordered and which ones you can afford to lose.

Edit: Also consider following these articles for implementing fast paced multiplayer. It's quite easily done with a powerful library like RakNet.