Model gets distorted when rotating the camera

Question

I'm currently developing my own 3d graphics engine and I'm having a hard time figuring out why my 3D models gets distorted when rotating the camera around.

This is my projection matrix. I'm following the OpenGL's Model.

def get_projection_mat(aspect_ratio, camera):  
    fov = camera.fov  
    z_near = camera.z_near  
    z_far = camera.z_far  
    top = math.tan(fov * 0.5) * z_near  
    bottom = -top  
    right = top * aspect_ratio  
    left = -right  
    projection_mat = np.identity(4, dtype=float)  
    projection_mat[0, 0] = 2 * z_near / (right - left)  
    projection_mat[0, 2] = (right + left) / (right - left)  
    projection_mat[1, 1] = 2 * z_near / (top - bottom)  
    projection_mat[1, 2] = (top + bottom) / (top - bottom)  
    projection_mat[2, 2] = -(z_far + z_near) / (z_far - z_near)  
    projection_mat[2, 3] = -2 * z_far * z_near / (z_far - z_near)  
    projection_mat[3, 2] = -1  
    projection_mat[3, 3] = 0  

    return projection_mat  

This is my view matrix:

def camera_matrix(self):  
    camera_matrix = np.identity(4, dtype=float)  
    camera_matrix[:3, 0] = self.left[:]  
    camera_matrix[:3, 1] = self.up[:]  
    camera_matrix[:3, 2] = self.forward[:]  
    camera_matrix[:3, 3] = self.pos[:]  

    return camera_matrix  

This is how I get the ViewProjection matrix:

    projection_mat = get_projection_mat(self.aspect_ratio, self.camera)  
    view_mat = self.camera.camera_matrix()  
    view_mat = np.linalg.inv(view_mat)  
    self.view_projection_mat = np.dot(projection_mat, view_mat)  

And, finally, this is the ViewProjectionModel matrix:

    view_projection_model_mat = np.dot(model.transform_mat.T, self.view_projection_mat) 

This is the method that I use to rotate the camera:

def rotate(self, yaw, pitch, degress=True):  
    rotation_mat_y = helper.rotate_matrix_y(yaw, degrees)  
    rotation_mat_x = helper.rotate_matrix_x(pitch, degrees)  
    rotation_mat = np.dot(rotation_mat_y, rotation_mat_x)  
    self.forward = np.dot(np.insert(self.forward, 3, 1), rotation_mat)  
    self.forward = helper.normalized((self.forward / self.forward[3])[:3])  
    self.up = np.dot(np.insert(self.up, 3, 1), rotation_mat)  
    self.up = helper.normalized((self.up / self.up[3])[:3])  
    self.left = np.dot(np.insert(self.left, 3, 1), rotation_mat)  
    self.left = helper.normalized((self.left / self.left[3])[:3])  

A video demo, showing the problem

To summarize, the order of multiplication is ProjectionMat X ViewMat x ModelMat x ModelFaces. If it isn't clear by now, I'm using Python for this project. Any help would be greatly appreciated.

With best regards, João Pedro

EDIT: Like this?

    directions, r = np.linalg.qr(np.array([self.forward, self.up, self.left]))  
    self.forward = directions[0]  
    self.up = directions[1]  
    self.left = directions[2]  

Answer 1

I see two potential errors here:

1. You're transforming directions like they're points.

   np.insert(self.forward, 3, 1) says "Add a homogenouse coordinate of 1 to this direction vector so that matrix translation affects it" - but you absolutely do not want translation to affect your directions! Turning 30 degrees is turning 30 degrees, no matter whether I do it at the origin or 50 km away from the origin.

   Your translation is already tracked by the pos variable, so don't let it leak into your directions this way. Instead, your 4th component / "w" of a direction vector should be zero, to ensure you factor in only the rotation of the vector.

   This also makes the division by w unnecessary, though you were already obliterating its result by normalizing it anyway.

2. You're allowing rounding errors to accumulate

   forward, up, and left are used as inputs to compute their own successors, meaning that any error in the input gets replicated in the output, along with any new rounding errors from the latest multiplication / normalization. So each small incremental error adds up and gradually snowballs up to a perceptible scale.

   You're re-normalizing the vectors, so you're correcting any errors in their lengths, but not their directions. So as the errors build up, the three basis vectors drift from being fully orthogonal, causing the skewing artifacts you're seeing.

   To fix this, you'll want to orthonormalise your basis to correct both magnitude and directional errors and stop them from building up. One way to do this is using the Gram-Schmidt process. You can compute the left vector from the cross product of up and forward to get a mutually-perpendicular unit vector by construction with a bit less work.