The trick here is to think of your image plane as though it was actually a flat object sitting in front of the camera, perpendicular to its view direction. This parallels the way the player's screen is a flat object in front of their eyes, perpendicular to their gaze - and this similarity is what makes the optical illusion of linear perspective work.
Following this line of thinking, each column of pixels on your screen corresponds to a specific point on this image plane in your game space.
Because the pixels on our screen are spaced out evenly, we need to space out these samples evenly in space along the image plane - I've visualized that with the line of little squares in the diagram above. So we need to find a direction to shoot each ray so it first "through" one of these evenly-spaced pixel points.
If we sweep our rays using equal angular increments like you do in this code:
angle = (w->cam->angle + (w->cam->fov / 2) - (i * w->cam->fov / WIDTH_MM));
then we don't get evenly-spaced points on our image plane - they'll bunch up in the middle and spread out at the sides, breaking our illusion. It makes straight edges of walls start looking bent.
In reality, the angular gap between adjacent pixels changes as we sweep our ray across the screen - with smaller angular spread at the sides and bigger strides in the middle:
So, working in angles can be a bit of a trap, and it helps to get out of angular space and into cartesian coordinates & vectors early. As a bonus, this will also save us most of our trig calculations! :)
First we'll need a unit vector in the direction the camera is facing, our forward
vector. We'll use this a lot, so it's worth saving rather than looking up the trig values every time:
forward.x = tcos(w->cam->angle);
forward.y = tsin(w->cam->angle);
Next we need a vector pointing parallel to the image plane, to the camera's right. We can use a little trick to compute this from forward with no more trig:
right.x = forward.y;
right.y = -forward.x;
And we'll need to now how wide to sweep from the center of our view to one edge.
halfWidth = ttan(w->cam->fov / 2)
Now the ray through the leftmost pixel in our image plane points along forward - halfWidth * right
, and the ray through the rightpmost pixel points along forward + halfWidth * right
.
We can calculate the direction the i
th ray should travel like so:
offset = ((i * 2.0 / WIDTH_MM) - 1.0) * halfWidth;
rayDirection.x = forward.x + offset * right.x;
rayDirection.y = forward.y + offset * right.y;
You can optionally normalize or scale this direction vector as needed to suit your ray intersection routines.