To implement this effect, there are a few things you need to know:
- The exact physical measurements of the display surface.
- The viewer's eye location, relative to the centre of the screen.
Once you have this information, it's relatively simple to get the effect. Instead of calculating a normal projection matrix (typically by taking the camera position, vertical field of view, aspect ratio, and near and far clip distances), you instead construct your own skewed projection matrix using explicit frustum planes.
To generate these explicit frustum planes, you use the centre of the screen as your "zero" position, and define each frustum plane by the viewer's eye location (relative to the zero position) and two of the corners of the screen (also relative to the zero position). Near and far clip planes may either be parallel to the screen (which is a little easier mathematically) or perpendicular to the eye->zero vector.
I seem to recall reading that D3D has an explicit API for creating skewed projection matrices, while you need to do the math yourself in OpenGL. Either way, there's plenty of sample code for constructing skewed projection matrices from arbitrary frustum planes in Google.
Once you have your skewed projection matrix, all the rest of your rendering happens normally, exactly as it does with a non-skewed projection matrix. Note: do not attempt to rotate or translate a "camera" object to mimic the viewer's motion; that's all handled explicitly by the skewed projection matrix. (If you wish the viewer to be able to navigate through the world to a greater extent than they can by moving physically, say by using a joystick, then you'll need to use a camera. Remember that any such camera motion/rotation would be relative to the physical screen, not relative to the viewer's perspective, so you'd need to do some extra maths if you wanted to make the camera move relative to the viewer's facing)
Anyhow, the net effect is that the game renders a carefully distorted view of the scene, precisely calculated to not appear distorted when viewed specifically from the viewer's eye position (and nowhere else).
A few notes on the video linked in the question. First, it looks like the effect being shown in the video has the helicopter placed slightly forward of the screen. Objects forward of the screen are always tricky to do well in this sort of system, as the view frustum is so narrow that it's so easy for them to wind up falling outside of the frustum when viewed from a sharp angle. For this reason, it's much more standard to place objects behind the screen, treating the screen as if it was a limited "window" into another 3D scene. If you're going to put objects in front of the screen (or intersecting the screen), then it's important to make these objects small relative to the size of the screen, and not to place them very far forward of the screen, or you risk users walking past them, or otherwise having difficulty finding them.
Also note that in this sort of rendering approach, the object will not appear to 'pop out'; it distinctly remains a 2D image on a flat screen. It's kind of disconcerting the first time you see something doing this in real life, rather than in a YouTube video; it's not at all what most people seem to have been expecting, despite intellectually knowing that the image is still just an image on a flat screen.
Finally, note that display lag is a serious issue with this sort of effect. Your whole architecture (viewer eye detection->rendering->display) needs to be as responsive as possible in an effort to minimise motion sickness among people using the system. This is a problem suffered by every VR-like system which tries to feed people images based upon their movements; many (but not all!) people simply can't cope with more than a few milliseconds of lag, and soon begin to feel nauseous. The longer the lag, the more people seem to be affected. So keep this in mind, too.