I'm building a display wall demo application.

The environment:

  • N machines, each driving up to six displays
  • Wired gigabit switch
  • OpenTK application to draw the pretty pictures, running 1 copy on each node

Each node runs an identical copy of the demo executable. Each node also has some config data containing the resolution of the entire wall and the relative position of that node's displays within that space. When rendering, a scale/translation transform is applied after the projection matrix (effectively zooming in on the part of the view that the node is responsible for rendering).

The nodes have a "mode" switch set in a startup script that puts them in "send" or "receive" mode. All but one node is in "receive." (Switching a node to "send" causes all others to switch to "receive").

The demo app is a basic FPS-type fly through of some primitive geometry (grid, icosahedron, etc). On each update, the sender node broadcasts a UDP datagram containing some state information (camera translation/rotation, transforms for moving objects).

Listeners asynchronously receive these datagrams, deserialize the state and update their local copies.

The net effect is that I run the app on my laptop and fly around, and the wall nicely follows along. This all works pretty well.

However, I'm concerned that as the system scales (from 10 displays to, say, 500), that network latency will be an issue. I've also thought that, rather than spamming state across the network this way, it would be nice to stuff it into a database and have each node just pull from there, but I doubt any off the shelf databases are going to be up to a 60FPS * 500 nodes cycle.

Suggestions? What's the best way to share state between a lot of nodes really, really fast?


1 Answer 1


Sounds like a job for multicast. (Sounds like you're already doing that?)

Certainly a database pull is a terrible idea for this kind of scenario. You still have to send out the same amount of data - but now you have a lot more overhead!

Really the problem you have to worry about, more than lag, is synchronisation. You want all the displays to match each other - even if they are all many milliseconds behind the actual simulation.

In particular, a database will exacerbate the problem by serialising your updates to clients (in the "not parallel" sense of the word). Each update must be sent to clients one-by-one. If you pump out an update's UDP packets in a burst, they should stick fairly close together. Database requests will introduce potentially huge timing gaps. (Multicast moves the problem of serialisation out to the router - which may or may not be a good thing.)

Of course, your client should have some kind of "lag buffer" - where it saves up some number of input frames and then interpolates between them for output. Normally this allows games to cope with network jitter and dropped packets.

With a lag buffer, this then becomes purely a clock synchronisation problem: As long as all of your clients can select the exact same frame within the lag buffer to display, they will all appear to be in-sync.

Synchronising your client's clocks may be as trivial as using NTP from a local network source and using the real time to select the frame to play back. Wikipedia suggests accuracy of 1ms on a LAN - more than good enough. But if that doesn't work, you may have to create your own clock-synchronising protocol.

  • \$\begingroup\$ Thanks for the advice. I'm using UDP multicast at the moment, though I suspect the router may be serializing packet transmission. I'd considered adding a local NTP server, then including the real transmission time in each packet. The clients would be configured with a desired latency, and use the time from the NTP server at render time, adjusted for the buffering latency, to interpolate between received states in their queue. Seems pretty straightforward. Thoughts? There are some slight synchronization issues \$\endgroup\$
    – 3Dave
    Commented Mar 7, 2013 at 15:01
  • \$\begingroup\$ Not sure what to add, at this stage. Perhaps you can make a prototype app that displays a predefined animated scene based on real time (just a simple clock would work), without having to do the complicated network state stuff, to see if NTP will work for you. \$\endgroup\$ Commented Mar 8, 2013 at 13:25
  • \$\begingroup\$ Added a "time line" store and a time server. Works great! I owe you beer. \$\endgroup\$
    – 3Dave
    Commented Apr 23, 2013 at 23:01
  • \$\begingroup\$ Thrilled to hear it works :) If you can, you should post a demo to YouTube or something :) \$\endgroup\$ Commented Apr 24, 2013 at 0:08
  • \$\begingroup\$ Will do - I'm gradually writing this up and building a free-standing example app. I'll add a link here when it's done (when I get time - which looks to be about the 5th of Never-1). \$\endgroup\$
    – 3Dave
    Commented Apr 24, 2013 at 13:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .