# Synchronizing over UDP

I have two apps communicating over UDP. One of the apps is a remote control and the other is a robot.

When the robot has some new data, for example a new sensor reading or GPS coordinate, I simply put this in an UDP packet on it's way to the remote for display and that is that. Since the sensor data has a short time span of validity it does not matter when the packet is lost, because a new one will follow with more updated data shortly anyways.

Similarly when the remote wants the robot to change behavior, I put the new desired behavior in an UDP packet on its way to the remote for execution. The only difference now is that this time I also send an "acknowlegement" packet back to the remote so that it will know that the the command was well received. Still this is a simple case.

However in the case where there is a state in the robot that BOTH the robot itself and the remote may change, for example if there is a "panic button" that can both be operated from the remote and the robot, how can the state be kept in sync? Since there is no guarantee the UDP packets will reach their destinations, I imagine there needs to be set up some kind of acknowledgement scheme that does not effectuate a command unless it knows that both remote and robot know what is about to happen. But to me this seems impossible to accomplish.

When the user presses the button to enter panic mode, generate a sequential ID to tie to that event of wanting to enter panic mode.

You then periodically send a message to the robot "Activate Panic Mode" with that sequence ID value you generated.

Whenever the robot receives an "Activate Panic Mode" message, have it respond with "Panic Mode Activated" with the same ID sent back. It can actually start doing panic mode behavior the first time it receives this message.

On the remote control side, keep sending that "Activate Panic Mode" message until you receive a message from the robot that it has entered panic mode, at which point you can stop sending them.

This works because the robot doesn't need confirmation from the remote control that it got the message, so you break the chain of infinite confirmation messages needed.

If you have a panic mode indicator on the remote, it's up to you if you want to activate that indicator as soon as the user presses the panic mode button, or if you want to wait til you get confirmation of panic mode from the robot.

Hopefully you can see how you'd do the same thing for deactivating panic mode once it's activated!

The sequence ID may be over kill, but I'm betting it will come in handy for dealing with more complex logic issues.

Lastly, if you are worried about facing twice the amount of unreliability, by having the remote periodically ask the robot for confirmation, you could have the robot send 2 or more confirmations in response to an "Activate Panic Mode" message, instead of just the one. Each side of the communication should be able to gracefully handle 0 to N copies of duplicate network messages with the same sequence IDs.

A common solution I use to send safe messages over UDP is the following :

When the state is changed on one side, send the message periodically (the more important, the shorter the period) until the receiver sends it back to you (or sends an acknowledgement). On the receiver side, send the acknowledgement everytime you receive it. This way, it is 100% reliable and as fast as UDP in most of the cases.

• How do you resolve the case when two changes happen at same time? Just compare timestamps? – Lennart Rolland Nov 29 '16 at 9:20
• You mean on the two connected devices ? – An intern has no name Nov 29 '16 at 9:22
• I mean if robot and remote changes the same setting to different values simultaneously – Lennart Rolland Nov 29 '16 at 9:23
• It depends on what strategy you use, but in this case, instead of comparing the time stamps, I would rather instorate a hierarchy. For example, the remote has the priority, and the robot could only change it when the remote allows it. The question is : what would you like to happen if the two change it at the same time ? Who has the priority ? – An intern has no name Nov 29 '16 at 9:26

Thanks to ideas in the other answers I have been pondering this and came to the following protocol:

1. Keep two copies of the parameters to sync in a registry in both robot and remote memory.
2. Whenever one node initiates a change to a parameter, the local copy is changed instantly and a stream of UDP packets is sent to the other node until it responds with an ack for the change.
3. Whenever the other node initiates a change to a parameter, the remote copy of the parameter is changed once the UDP packet arrives, before ack is returned.
4. Whenever there are NO changes, send a continuous stream of UDP packets indicating "no change".
5. Each parameter in both the local and remote copies of the registry holds in addition to their value, a timestamp of last confirmed sync (for local, it's the last received ack, for remote its the last change), and the value is considered confirmed when less time than the estimated UDP round trip time + margin has passed since the sync was recorded.
6. Each UDP packet for both change and ack will have a serial number to allow multiple packets of the same serial to be ignored.

So you might ask with two copies of the parameters on both sides, how do we know what value should be used? The "best" value for each parameter is simply then the one with the last timestamp, however it will be up to the application to decide what to do with the extra "uncertainty" and "knowledge about the other side" data, and to decide when to use which value in the registry.

For example in the panic button case, the robot should be conservative and say as long as at least one of the panic values is TRUE then the time since sync does not matter, panic will be enabled either way.

Another example would be robot throttle, which would be lowered or even completely stopped in the case of lost sync (to avoid the robot running away).

In the case of UI in the remote, it can give all sorts of useful feedback to the user such as "grayed out" when fallen out of sync, or a tiny bar graph that shows sync lag etc.

If we cannot afford such high bandwidth usage, we can slow down by simply sending packets with higher delay while at the same time expect slower round trip times.

This protocol will under good network conditions let both sides know that sync is confirmed with a delay of UDP round trip time + margin, and under bad network conditions they will know by the "lack of ack" that the sync status is unconfirmed.

It will also let both sides know what the other side thought to be the current value of each parameter last time the network was in good condition.