C++ Mouse picking for ray to plane intersection

Question

I'm making a C++/OpenGL application (decidedly without GLM), and I'd like to understand how to create a ray from the camera eye to the mouse pointer, as well as detect an intersection point from the resulting ray onto a flat plane.

I have: - Camera position, camera pitch/yaw, camera's Fwd vector - mouse coordinates in pixel measurements (which I can convert to -1:+1 screen space) - A world system where XY are the horizontal and Z is up - A flat plane where Z=0 - My own matrix class which is basically just a float[16] and accessed (Get()) as a float* - Basic matrix multiply function, cross, dot product functions.

I'm still trying to learn 3D math (this project is somewhat of a reason for that), so I'm still somewhat hazy on how to accomplish this. Most tutorials I can find are Unity tutorials, or involve using an Unproject function from glu or GLM.

I'd hopefully like to understand how to accomplish this manually as I'd like to understand what's happening. Thank you for any wisdom or guidance.

Answer 1

Ok: Math hat on:

In computer graphics/games, you typically work in a couple of frames of reference:

1. local/object space
2. world space
3. view space
4. screen space

Local space is represented by relative to the origin of whatever 3d mesh you want to draw.

World space is relative to the game world, i.e. the mesh translated to a position in the world, and rotated to an orientation within the world.

View space is relative to the camera, i.e is it behind, in front of, above, below etc the camera, and by how much. Think of it this way: I Tell you to meet me at the entrance of the Empire State building in New York. That is the world position. From my perspective (view space), it is simply two metres in front of me.

Screen space is when everything is transposed onto the 2d screen and scaled down for display.

So, to get from local space to screen space we have to apply what are known as affine transformations, using 4x4 matrices.

The basic construction of a transform matrix is as follows:

[ R    U     F    T]

[ XR   XU    XF   X]
[ YR   YU    YF   Y]
[ ZR   ZU    ZF   Z]
[ SX   SY    SZ   0]

Ok, so some explanation of the columns is necessary: R stands for "Right", and represents one principle axis of the frame of reference. U stands for "Up", F for "Forward", and T for "Translate". Never mind about S, because it will just muddy the waters at this stage.

Right, up and forward are perpendicular axes, but also represent a change in orientation which will occur when applied to a point, or vector (important later!). Translation just means how much to move the point or vector.

So, now that we have this information, we can now start moving stuff around.

local space is usually defined thusly:

[ 1   0   0  0]
[ 0   1   0  0]
[ 0   0   1  0]
[ 1   1   1  0]

And is known as the "identity matrix"

Basically, if we multiply a vector by this matrix, it will be unchanged.

So, we construct a world matrix, based on the desired position and orientation in the world. If you want to know how to do this, I would recommend reading up on GLM, or whatever math library you want to use for specifics.

Then we construct a "view matrix", which represents the position, direction, and orientation of the camera, or eye. Again, math library specifics can vary, so choose a library and go with the API.

Finally, you need a projection matrix. This is usually comprised of resolution based information etc. Again, API's vary.

Now, to get a point from local space into screen space, we multiply the vector by each of the matrices in turn:

transformed_point = world_transform * point;
transformed_point = view_transform * transformed_point;
transformed_point = projection * transformed_point;

Or: transformed_point = projection * view_transform * world_transform * point;

No, to your question:

in order to transform a 2d point, into a 3d point, you must perform the transform in reverse, using the inverse of the matrices:

world_point = inverse(view) * inverse(projection) * mouse_point;

Then, you can draw a line between the camera position, and the normalised(length of one) mouse_point vector * ray length.

Some resources:

http://antongerdelan.net/opengl/raycasting.html

http://schabby.de/picking-opengl-ray-tracing/

http://www.opengl-tutorial.org/

I recommend the third link, and in particular, read the maths stuff. You'll need it.