# How do you triangulate a 3D point from a pair of stereo images?

I'm working with a stereo pair of photos, and trying to obtain a set of 3D points from matching points on these 2 images. How exactly is the point triangulated? Do you throw a ray from the camera pinhole centerpoint? Or do you throw a ray perpendicular to the photos?

Edit: I'm also looking for code examples, that document the simple linear/polynomial math involved. Currently I'm simply using 3D line/line intersections to find the point. (actually line/plane intersections) I'm sure this is not accurate and the calibration data of the camera should also be used. Although I don't know how. For instance, with the points on the photos known, should I be drawing the 3D line from the camera pinhole 'origin', or is there a better way? Should I be taking the lens distort into account, or something fancier? The calibration data I have is as follows:

• ix, iy -- Image sensor width and height, i.e. the dimensions of the photograph.
• fx, fy -- Focal length in x- and y-dimensions measured in pixels.
• cx, cy -- Principal point coordinates, i.e. coordinates of lens optical axis interception with sensor plane.
• skew -- Skew transformation coefficient.
• k1, k2, k3 -- Radial distortion coefficients.
• p1, p2 -- Tangential distortion coefficients.
• Are you using OpenCV for this? Dec 23, 2011 at 16:59
• No existing library. Just a bit of my own math within my own UI. Dec 24, 2011 at 9:24
• Dec 24, 2011 at 11:46
• It might be worth looking into the OpenCV implementation then. Especially the part about Camera Calibration and 3D Reconstruction Dec 24, 2011 at 11:55

The first image is more correct, because image on the photo is perspective rather than orthographic. It can be orthographic on macro photos. Possibly it depends on what type of lens do you use.

• Updated question, can you help some more? Dec 23, 2011 at 16:39
• Sorry, but I cannot, I've never worked on such theme, but from my experience in 3d programming, it looks like you have to build your projection matrix taking into account that parameters. And then use something(google on this if you haven't met it, "picking objects in 3d") like Unproject and you'll get a picking ray. Dec 23, 2011 at 17:37