Ray Tracing in 3D - generating a ray

Question

A ray is shot from the camera eye to the camera screen at point (i,j) how do I get the coordinates of that point in 3D world coordinates?

The best I could find online is for point S S = Q + au + bv

Q being the camera direction and u,v are the UV plane of the camera.. not sure how to find their coordinate and if I need them in world coordinates or something else.

If you could clarify this issue, it would be great.

Thanks.

Answer 1

You can calculate the ray from camera for any pixel px,py with the following formula:

ray.origin = camera.getPosition();
relativeX = px * 2 / screenWidth - 1; // [0, screenWidth] -> [-1, 1]
relativeY = py * 2 / screenHeight - 1; // [0, screenHeight] -> [-1, 1], might need to be flipped depending on the coordinate system
ray.direction = vector(relativeX * sin(FOV_X / 2),
                       relativeY * sin(FOV_Y / 2),
                       1);
ray.direction.normalize();
camera.getOrientation().transform(ray.direction);

This creates the rays first with identity camera and then transforms it according to the camera orientation. If your camera is a 4x4 matrix, camera.getPosition() should return the fourth column/row from the matrix and camera.getOrientation() should return the 3x3 upper left part of the matrix. If you have ModelToView matrix, then camera matrix is the inverse of that. If the 3x3 part of your camera matrix has only rotation but no scaling, it's simply transpose of the matrix.

A more general approach is to invert the projection matrix also. For example vertex shader might be transforming coordinates with:

position = projection * worldToCamera * modelToWorld * vertexPosition;

modelToWorld matrix is not relevant in this case, so basically this is the same as:

position = projection * worldToCamera * worldPosition;

Which can be unprojected by

worldPosition = inverse(worldToCamera) * inverse(projection) * position;

or

worldPosition = inverse(projection * worldToCamera) * position;

If you unproject two vectors with different screen space z values (e.g. px,py,-1,1 and px,py,1,1), you get two vectors in world space, which defines a line in 3D space for the ray you are looking for.