# Efficient data packaging for a client-server network

Language: C++

My question is as follows: I would like to know what would be the best or at least a good way to pack and send data from client to server and the reverse. There will be a few data composing a single packet. A packet will have a "id", that defines what is it for, then the data in a pre- determined order for that "action" which the packet corresponds.

For less performance-dependant systems, i would just send strings, that would be separated by a space, being them the data of the "action"and the first "word" the packet identifier and just chain if statements checking when there is a match.

Now for a more critical system, what i tought so far was something like this:

Make a string with packet id and data, and send it. Then, to unpack, i could extract the first integer in the string, and by having an array of packet handlers, with indices corresponding to the packet id they handle, and just do something like packetHandlers[packetID].Process(packetData) .

What do you think of it, any suggestions? greatly apreciated!

First, confirm you actually need a a fancy, efficient protocol before wasting resources developing it. Don't forget debugging/modifying your game will be more difficult and time-consuming due to your fancy protocol. I would just abstract the network communication away so the actual implementation can be easily swapped for a more efficient one if needed in the future. Use the simplest protocol possible until you hit performance issues. Another advantage of designing your protocol later is the protocol can be optimized for actual data transported vs. what you predict will be transported.

After confirming you need a fancy protocol, look at protocols that others have spent considerable time developing. Some examples:

• (update) The original developer of Protocol Buffers (v2) developed a new protocol called Cap'n Proto. He explains his design decisions and compares to other similar libraries that have recently been released: Cap'n Proto, FlatBuffers, and SBE.
• Google's main bottleneck is network communication between computers, so they probably considered efficiency when developing Protocol Buffers. Gracefully handles forwards/backwards compatibility (when you decide to alter your data structures). Used by every major Google product (Gmail, Search, etc)
• Apache Thrift is a similar protocol used by Facebook.
• RakNet is an open-source network library specifically designed for game development.
• ZoidCom is another networking library geared towards game development. It's not open-source, but you can still study it for design hints.

The First Rule of Program Optimization: Don't do it.
The Second Rule of Program Optimization (for experts only!): Don't do it yet.

[Michael A. Jackson]

In other words: your primary optimization parameter should be: life (years of life per program implementation). [How to Program Independent Games. Slide 21. Jonathan Blow.]

• nice reading man! thanks a lot for the complete answer, it sure helped, and im following your advice entirely, just abstract this, making it simple enough for now :) – Grimshaw Jul 21 '11 at 22:22
• Pretty sure that Google's use case for protocol buffers wasn't efficiency in the standard sense, but to maximize compatibility between all possible platforms and any future data versions (which is efficiency in another light). I'll be over here reading up on your other notes, very nice collection for me to get reacquainted with the subject. – Patrick Hughes Jun 23 '14 at 1:57
• RakNet is not open source. – Gerstmann Jun 23 '14 at 13:06
• @Gerstmann: Raknet is open source (once again): github.com/OculusVR/RakNet – Leftium Jul 7 '14 at 20:24

Why use two different encoding schemes? Just use the second one for every system. Just keep it simple.
Consider using delta compression. I.e. send one full value and after that only the things that changed. After a few game iterations send a full value again.
Another encodig you could consider is Base 128 Varint. Google Protobufs use it. Have a look at the "Encoding" page of their developer guide: Protocol Buffers Encoding Could save a few bytes.

• The first system i spoke of was a mere example from another projects :) I certainly enjoyed the delta compression idea, thanks mate! – Grimshaw Jul 20 '11 at 23:36

What might be an example of the data you're sending? I don't see any reason to do anything overly fancy. Once the data is fully loaded into the receiver's buffer, inspect the first int based on its value, you then know how to process the rest of the data.

So a packet that has four data parts id, val1, val2, and val2 might look like this:

// I'm using words (one byte) so my sample data is short
00000001 00101000 00010110 00010100


As you read the first byte (which you know will always be there) you decide how to process the next set of data. If the first word (id) is 00000001 you know there are three more dwords following, and that is the end of the packet. To continue the example, you might have id = 00000010 and your specification tells you that for id value 2, you process float, float, float in that order, which might specify a player position in world space.

Think of it as writing your own binary file system, you have a header value, which describes the rest of the data, where it is located (what position) and what type of data to treat it as.

• Got it, the only question i remain is: is packing all those ints and floats in a standard string enough, or should i go with lighter data types? – Grimshaw Jul 21 '11 at 22:25
• As said by others, start with the string and see what happens. Chances are, you'll have a bottle neck elsewhere in your code. – Nate Jul 22 '11 at 15:57