2
\$\begingroup\$

A ray is shot from the camera eye to the camera screen at point (i,j) how do I get the coordinates of that point in 3D world coordinates?

The best I could find online is for point S S = Q + au + bv

Q being the camera direction and u,v are the UV plane of the camera.. not sure how to find their coordinate and if I need them in world coordinates or something else.

If you could clarify this issue, it would be great.

Thanks.

\$\endgroup\$
4
  • 1
    \$\begingroup\$ That is called 3d picking, and you ought to be able to find some good explanations already posted. gamedev.stackexchange.com/search?q=3d+picking \$\endgroup\$ May 14, 2013 at 15:37
  • \$\begingroup\$ are you sure? what im trying to do is to figure out how to create a ray that goes through a pixel... how to find the position of that pixel in 3d space \$\endgroup\$
    – Aviran
    May 14, 2013 at 15:54
  • \$\begingroup\$ You are trying to check for intersections? Against what? Spheres? Cube? Triangle? \$\endgroup\$
    – olevegard
    May 14, 2013 at 16:12
  • 1
    \$\begingroup\$ "Unproject" is what you want. It's typically part of 3D picking and it converts screen space to world space. \$\endgroup\$
    – House
    May 14, 2013 at 17:42

1 Answer 1

5
\$\begingroup\$

You can calculate the ray from camera for any pixel px,py with the following formula:

ray.origin = camera.getPosition();
relativeX = px * 2 / screenWidth - 1; // [0, screenWidth] -> [-1, 1]
relativeY = py * 2 / screenHeight - 1; // [0, screenHeight] -> [-1, 1], might need to be flipped depending on the coordinate system
ray.direction = vector(relativeX * sin(FOV_X / 2),
                       relativeY * sin(FOV_Y / 2),
                       1);
ray.direction.normalize();
camera.getOrientation().transform(ray.direction);

This creates the rays first with identity camera and then transforms it according to the camera orientation. If your camera is a 4x4 matrix, camera.getPosition() should return the fourth column/row from the matrix and camera.getOrientation() should return the 3x3 upper left part of the matrix. If you have ModelToView matrix, then camera matrix is the inverse of that. If the 3x3 part of your camera matrix has only rotation but no scaling, it's simply transpose of the matrix.

A more general approach is to invert the projection matrix also. For example vertex shader might be transforming coordinates with:

position = projection * worldToCamera * modelToWorld * vertexPosition;

modelToWorld matrix is not relevant in this case, so basically this is the same as:

position = projection * worldToCamera * worldPosition;

Which can be unprojected by

worldPosition = inverse(worldToCamera) * inverse(projection) * position;

or

worldPosition = inverse(projection * worldToCamera) * position;

If you unproject two vectors with different screen space z values (e.g. px,py,-1,1 and px,py,1,1), you get two vectors in world space, which defines a line in 3D space for the ray you are looking for.

\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .