How to avoid being throttled?

Question

I'm writing a networked iOS game. When sending packets with GKMatchSendDataReliable (which I assumed was UDP with their own packet reception code written) at 60 packets per second (so 16 ms between adjacent packets), average ping times rapidly get worse: I opened 7 GameCenter matches below (one after the other) and simply sent a "flood" of 100 packets (at a rate of 60 packets per second). I measured the average roundtrip time, and these are the results:

[ 21:16:39 ]:  I saw an average roundtrip time of 52.342787 ms, he saw 54.496590 ms
[ 21:16:34 ]:  I saw an average roundtrip time of 62.631942 ms, he saw 61.991655 ms
[ 21:16:45 ]:  I saw an average roundtrip time of 88.394380 ms, he saw 83.619123 ms
[ 21:16:51 ]:  I saw an average roundtrip time of 179.053118 ms, he saw 156.869141 ms
[ 21:16:57 ]:  I saw an average roundtrip time of 75.025076 ms, he saw 75.419723 ms
[ 21:17:23 ]:  I saw an average roundtrip time of 8832.082488 ms, he saw 7616.877558 ms
[ 21:19:33 ]:  I saw an average roundtrip time of 25088.962344 ms, he saw 16833.064914 ms

After the last 2 tests the results are around 1000 ms.

It would seem that I'm being throttled, most likely by my ISP. Because this is an iOS game, people will use regular residential connections.

When I changed the packet send rate to being 10 times slower (so 1 packet every 160 ms), the tests take much longer, but roundtrip times remain consistently low.

[ 21:31:27 ]: I saw an average roundtrip time of 55.289109 ms, he saw 69.032727 ms

So it looks like to keep low latency on the connection (and not be "punished" by ISPs) I have to reduce the rate of packets that I send. Keep in mind these are very small packets, like 40 bytes maximum, yet I'm still being throttled.

I'm looking for guidelines on how many UDP packets I can send per second to avoid being throttled! Are there any general guidelines anywhere?

Answer 1

Not even home PC based action games or big MMOs run their packets at 60Hz. Plus having really small packet sizes is not necessarily a great thing, each one of those tiny packets has a big overhead in just sending it out.

Try shooting for 10Hz updates with some client-side interpolation. I assume that you're already interpolating because there will always be ping delays.

Read up on MTU sizes and throwing more info in there to cover the longer time frame. An average packet size at the transport layer will be 1400 or so, anything over the MTU size will split your message and cause even more overhead.

Answer 2

First, you have to make sure how big the entire data is. Your ISP will most likely care about actual bytes sent, not the amount or frequency of datagrams. If you're sending maximum (65507 payload octects) sized datagrams 60 times per second, you're sending around 30 Mb/s upstream. Not everybody has that kind of connection.

Remember that the IP header is 20 octets long, and the UDP header is 8 octets long. That's an additional 28 octets you're sending for each datagram.

If you're not maxing out your connection, there are many places where your packets may be getting delayed: namely the client's OS, your gateway (probably a wireless router or cable modem), your ISP, the other peer's ISP, the other peer's gateway, the other peer's OS among others.

In case you haven't used it yet, I recommend you use Wireshark, which is an extremely powerful tool to diagnose networking problems. Think of it as the equivalent of a debugger, but for networking.

There are several ways you can diagnose network traffic with Wireshark:

• Use Wireshark in a PC in the same network as the mobile device, with a promiscuous hub and setting your network device as promiscuous

• Set a PC as a wireless gateway and connect your mobile device to that gateway, then listen with Wireshark on said PC

• Run Wireshark in the same machine as an emulator

• Run tcpdump in the device itself (easy on Android, requires jailbreak on iOS), and then read the captured data on Wireshark

• Make a simple program that does the exact same thing, but that works on a PC, and use Wireshark in there.

• ... and many others

You want to check which packages are getting sent, and when. For example, if the delay occurs before they're getting sent, you're being throttled by the OS; while if you're getting the delay even on a desktop version of the same program, this means you're being throttled by the network somewhere.

Usually, if you're getting throttled by the network, you should get ICMP type 4 datagrams, so you can use those to check where exactly you're getting throttled.

In conclusion, there are many reasons why your packages may be getting delayed, and it would be wise to find out where the problem is before you start attempting to solve the problem.

Answer 3

It appears one of my assumptions was wrong. According to this:

GKMatchSendDataUnreliable mode, the image to be transmitted in the so-called UDP. GKMatchSendDataReliable mode image send by TCP. Should it be a GKMatchSendDataUnreliable usually.

Changing the send mode to real UDP (ie GKMatchSendDataUnreliable) appears to maintain low ping rates at 60 packets per second. It looks like I've been struck by Nagles yet again.

I still get weird behavior (periods with very high ping times), but I'm not sure the root cause (ISP or network congestion).

[ 10:30:33 ]:  I saw an average roundtrip time of 39.908923 ms, he saw 48.437794 ms
[ 10:30:39 ]:  I saw an average roundtrip time of 26.278577 ms, he saw 27.023854 ms
[ 10:30:48 ]:  I saw an average roundtrip time of 23.254163 ms, he saw 24.495182 ms
[ 10:30:54 ]:  I saw an average roundtrip time of 37.333127 ms, he saw 34.780404 ms
[ 10:31:03 ]:  I saw an average roundtrip time of 29.198575 ms, he saw 29.071106 ms
[ 10:31:11 ]:  I saw an average roundtrip time of 49.030299 ms, he saw 48.675459 ms
[ 10:31:18 ]:  I saw an average roundtrip time of 34.031792 ms, he saw 34.698117 ms
[ 10:31:24 ]:  I saw an average roundtrip time of 30.058642 ms, he saw 32.814952 ms
[ 10:31:30 ]:  I saw an average roundtrip time of 53.110438 ms, he saw 54.271453 ms
[ 10:31:45 ]:  I saw an average roundtrip time of 119.693933 ms, he saw 107.616359 ms
[ 10:31:50 ]:  I saw an average roundtrip time of 222.644443 ms, he saw 229.589861 ms
[ 10:31:57 ]:  I saw an average roundtrip time of 166.827070 ms, he saw 167.647724 ms
[ 10:32:05 ]:  I saw an average roundtrip time of 765.356593 ms, he saw 859.600923 ms
[ 10:32:13 ]:  I saw an average roundtrip time of 357.522686 ms, he saw 339.648654 ms
[ 10:32:24 ]:  I saw an average roundtrip time of 1115.639593 ms, he saw 1060.574401 ms
[ 10:32:39 ]:  I saw an average roundtrip time of 175.845995 ms, he saw 171.112166 ms
[ 10:32:44 ]:  I saw an average roundtrip time of 47.262925 ms, he saw 41.987869 ms
[ 10:32:52 ]:  I saw an average roundtrip time of 74.524443 ms, he saw 78.868198 ms
[ 10:33:47 ]:  I saw an average roundtrip time of 20.943917 ms, he saw 21.217377 ms
[ 10:33:52 ]:  I saw an average roundtrip time of 28.944821 ms, he saw 29.303144 ms
[ 10:34:06 ]:  I saw an average roundtrip time of 25.581624 ms, he saw 25.439416 ms
[ 10:34:13 ]:  I saw an average roundtrip time of 25.565568 ms, he saw 25.655267 ms
[ 10:34:18 ]:  I saw an average roundtrip time of 38.609394 ms, he saw 37.462835 ms

Later:

[ 10:38:11 ]:  I saw an average roundtrip time of 40.037623 ms, he saw 43.367524 ms
[ 10:38:21 ]:  I saw an average roundtrip time of 121.222703 ms, he saw 118.855264 ms
[ 10:38:28 ]:  I saw an average roundtrip time of 726.391897 ms, he saw 685.742454 ms
[ 10:38:33 ]:  I saw an average roundtrip time of 60.251207 ms, he saw 57.974503 ms
[ 10:38:42 ]:  I saw an average roundtrip time of 1133.909392 ms, he saw 1124.404501 ms     

So it's sporadic and goes in waves. I guess I will have to try some of the suggestions in the other posts such as reducing my packet send rate.