I am starting to get my feet wet with game networking, having iterated many times now on some simple 2D games. For learning purposes, I've read the fantastic Gaffer on Games networking article a couple of times, the Source networking article, and the Quake sourcecode review about networking. I'm feeling comfortable with the concepts discussed there.
My question is about the proper implementation of data buffering within a game loop, with the intention of optimizing performance, and keeping IP fragmentation in mind. I'll give a python pseudocode example from just the client perspective, for simplicity. Also assume we are using UDP sockets.
Let's say hypothetically there are some events that can occur client-side, which should be transmitted to the server. Let's encapsulate this data in a simple dictionary called a message. A single message might look like this:
message = {
'type': 'move',
'data': 'right',
}
Within a single iteration of the client game loop, its possible for many of these messages to be generated. Here's an ultra-stupid game loop that might do this:
def main_loop():
if game.input.buttonA:
send_message({'type': 'foo', 'data': 'bar'})
for event in game.system.events:
send_message({'type': 'event', 'data': event.data})
game.render()
A naive implementation of send_message() would be to serialize the message and fire it off in a packet to the server immediately. The problem with that is that we could be potentially sending hundreds of packets per frame--I don't understand the low level details but I would think flooding the socket like this would be dangerous and bad.
Since these messages are really small when serialized, like <50 bytes, we could batch them up together in a single packet to send to the server. The implementation of that might look like this:
# Queue a message instead of sending it immediately
def queue_message(msg):
message_queue.push(msg)
# Flush the message queue into a single stream, JSON serialize and send to server
def flush_messages():
buf = []
for message in message_queue:
buf.append(json.loads(message))
socket.send(''.join(buf))
message_queue.clear()
def main_loop():
if game.input.buttonA:
queue_message({'type': 'foo', 'data': 'bar'})
for event in game.system.events:
queue_message({'type': 'event', 'data': event.data})
game.render()
flush_messages() # Send actual packet at the end of the loop
This seems much better, because we're sending far less packets per frame by grouping messages together. But what happens if the game loop generates a LOT of messages? Well, if the total packet size is over the MTU (around 1500 bytes for UDP), then we're going to experience fragmentation.
Lets say you send a 2500 byte packet. It will get fragmented into two separate packets and then reassembled transparently on the other end. From how I understand it, the problem with this is that if either half gets dropped, the whole packet is lost. When we're dealing with a highly realtime networking architecture like a game here, I think we actually would prefer to have only half of those messages and get the other half later, rather than none at all. Especially because the messages dont have any dependency on each other, they are just little chunks of data.
The problem I'm facing now is how to properly implement application level fragmentation, such that I can control how packets get split up. One way to do this would be to go through all my messages, putting them into packets until the packet is full, and then create a new packet, finally sending all of the packets at once.
def flush_messages():
buf = []
bufsize = 0
packets = [buf]
# Stuff messages into packet buffers, overflowing into the next when we hit MTU
for message in message_queue:
msgsize = len(message)
if bufsize + msg_size > MAXIMUM_PACKET_SIZE:
buf = []
bufsize = 0
packets.append(buf)
buf.append(message)
bufsize += msgsize
# Send all packets now and empty the message queue
for packet in packets:
socket.send(''.join(packet))
message_queue.clear()
Another way would be to simply only send the first 1500 bytes worth of messages this frame, and allow the rest to be handled the next frame.
def flush_messages()
buf = []
bufsize = 0
for message in message_queue:
msgsize = len(message)
if bufsize + msgsize > MAXIMUM_PACKET_SIZE:
break
buf.append(message)
# Send packet, but DONT clear message queue
socket.send(''.join(buf))
Of course, this latter approach has the danger of the buffer climbing over time and the server experiencing massive lag with respect to the client.
What's the right way to do this? Is my buffering approach here is wrong to begin with?
