Efficient data packaging for a client-server network

Question

Language: C++

My question is as follows: I would like to know what would be the best or at least a good way to pack and send data from client to server and the reverse. There will be a few data composing a single packet. A packet will have a "id", that defines what is it for, then the data in a pre- determined order for that "action" which the packet corresponds.

For less performance-dependant systems, i would just send strings, that would be separated by a space, being them the data of the "action"and the first "word" the packet identifier and just chain if statements checking when there is a match.

Now for a more critical system, what i tought so far was something like this:

Make a string with packet id and data, and send it. Then, to unpack, i could extract the first integer in the string, and by having an array of packet handlers, with indices corresponding to the packet id they handle, and just do something like packetHandlers[packetID].Process(packetData) .

What do you think of it, any suggestions? greatly apreciated!

Answer 1

First, confirm you actually need a a fancy, efficient protocol before wasting resources developing it. Don't forget debugging/modifying your game will be more difficult and time-consuming due to your fancy protocol. I would just abstract the network communication away so the actual implementation can be easily swapped for a more efficient one if needed in the future. Use the simplest protocol possible until you hit performance issues.

After confirming you need a fancy protocol, look at protocols that others have spent considerable time developing. Some examples:

• Google's main bottleneck is network communication between computers, so they probably considered efficiency when developing Protocol Buffers. Gracefully handles forwards/backwards compatibility (when you decide to alter your data structures). Used by every major Google product (Gmail, Search, etc)
• Apache Thrift is a similar protocol used by Facebook.
• RakNet is an open-source network library specifically designed for game development.
• ZoidCom is another networking library geared towards game development. It's not open-source, but you can still study it for design hints.

---

The First Rule of Program Optimization: Don't do it.
The Second Rule of Program Optimization (for experts only!): Don't do it yet.

[Michael A. Jackson]

In other words: your primary optimization parameter should be: life (years of life per program implementation). [How to Program Independent Games. Slide 21. Jonathan Blow.]

Answer 2

Why use two different encoding schemes? Just use the second one for every system. Just keep it simple.
Consider using delta compression. I.e. send one full value and after that only the things that changed. After a few game iterations send a full value again.
Another encodig you could consider is Base 128 Varint. Google Protobufs use it. Have a look at the "Encoding" page of their developer guide: Protocol Buffers Encoding Could save a few bytes.

Answer 3

What might be an example of the data you're sending? I don't see any reason to do anything overly fancy. Once the data is fully loaded into the receiver's buffer, inspect the first int based on its value, you then know how to process the rest of the data.

So a packet that has four data parts id, val1, val2, and val2 might look like this:

// I'm using words (one byte) so my sample data is short
00000001 00101000 00010110 00010100 

As you read the first byte (which you know will always be there) you decide how to process the next set of data. If the first word (id) is 00000001 you know there are three more dwords following, and that is the end of the packet. To continue the example, you might have id = 00000010 and your specification tells you that for id value 2, you process float, float, float in that order, which might specify a player position in world space.

Think of it as writing your own binary file system, you have a header value, which describes the rest of the data, where it is located (what position) and what type of data to treat it as.