Broken model matrix builder

Question

I'm trying to setup basic render pipeline in Vulkan and am currently setting up the perspective, view and model matrices.

The line that calculates the position in the vertex shader is this:

gl_Position = ubo.model * vec4(inPosition, 1.0);

where inPosition is a vec3, while ubo.model is a mat4 representing the model matrix

I'm not touching the camera or perspective yet, so the camera is in the origin looking towards the positive Z axis. This is the object I'm passing to the vertex shader:

const std::vector<float> vertices = {
    -0.25f, -0.25f, 0.0f,
    0.25f, 0.25f, 0.0f,
    -0.25f, 0.25f, 0.0f,
    0.25f, -0.25f, 0.0f,
};

const std::vector<uint16_t> indices = {
    0, 1, 2,
    0, 3, 1
};

Object initial m_position and m_rotation are set to glm::vec3(0.0f) and glm::quat(0.0f).

These are the transformations that I then apply to the object:

object->translate(glm::vec3(0.0f, 0.0f, 0.5f));
object->rotate(glm::vec3(glm::radians(5.0f), 0.0f, 0.0f));

First line pushes the object 0.5 toward the positive Z, moving it away from the camera. The rotate function actually takes a glm::quat, but passing in a vec3 interprets the vector as euler angles and casts them to an equivalent quaternion.

I've also bound the rotate call to be called once per frame, so I can observe how the object is rotated.

These are the translate and rotate functions:

void Mesh::translate(glm::vec3 offset) {
    m_position += offset;
}

void Mesh::rotate(glm::quat rotation) {
    m_rotation = m_rotation * rotation;
}

I'm honestly not sure if I'm multiplying the rotations in the right order, but it seems to work the same however I order it. I know that the quaternion multiplication is not commutative, but perhaps the change here can only be observed with a perspective.

This is the code that (currently) calculates the model matrix out of model position and rotation:

glm::mat4 Mesh::getMeshMatrix() {
    glm::mat4 meshMatrix(1.0f);

    meshMatrix = glm::mat4_cast(m_rotation) * meshMatrix;
    meshMatrix = glm::translate(meshMatrix, m_position);

    return meshMatrix;
}

m_rotation is a glm::quat and m_position is a vec3. I'm also not sure about the rotation multiplication order, but since the meshMatrix is an identity matrix, I don't think it makes a difference in this particular case.

If I comment out both of the lines, I get a square in the middle of my window, as expected (I set the window to have the aspect ratio of 1, at least until I introduce the perspective matrix).

If I comment out only the rotation line, I get the same output as earlier. The square is pushed back 0.5f, but since there's no perspective, it looks the same. I've tried changing the translate function call to move it left, right, up and down and it works as one would expect. I believe that the translation works as it should.

If I comment out only the translation, the square rotates around the X-axis (horizontally). Only one side of it is visible (due to face culling), and even then, I can only see the half of the square because either upper or lower half is now within negative Z-axis and is being clipped out. I can also change the call to rotate it around the Y-axis and Z-axis. Rotation along the Y-axis is analogous to the X-axis rotation, only the rotation is now vertical. Rotating it around the Z-axis spins the square in a way that it's visible all the time (like watching a carousel from above), which is to be expected since the z coordinates are not changed. So I conclude that rotation works as it should.

The problem arises when I comment out nothing. What I get is a square that is first translated along the Z-axis, and then the whole system is rotated around the origin. The result is what you'd see if you were in a rolling barrel with the square taped to the inside of it. As the square is rotated in the negative z it disappears due to the fact that it's behind the camera (and even if not for that, I still could not see it due to face culling).

This is what would happen if I were to translate the object before I rotate it. but no matter on how I order the two lines, I get the same result.

Once meshMatrix is returned from the getMeshMatrix, it will be copied to the GPU as is, with no further modifications, so I can't see and error happening there. And position and rotation are completely separated prior to that function, so I don't believe that anything fishy happens beforehand. And finally, translation and rotation work properly on their own, so their interaction must somehow be wrong. And only thing that can really be wrong is order of operations, but changing the order does not change the output, which seems wrong.

What gives? What am I missing?

Answer 1

I found what I messed up. It seems that glm::translate(mat4, vec3) doesn't do what I though it does.

What I thought it does is that it takes the provided mat4 and vec3, treats the mat4 as a transformation matrix, and applies the translation described by vec3.

It obviously works in some other way (or there is a detail I have missed that breaks it in the way I used it).

The function that actually does what I want is the following:

glm::mat4 Mesh::getMeshMatrix() {
    glm::mat4 identity(1.0f);

    glm::mat4 rotate = glm::mat4_cast(m_rotation);
    glm::mat4 translate = glm::translate(identity, m_position);

    return translate * rotate;
}

It's a bit more memory inefficient, but at least it works.

If anyone comes across this and figures out what exactly I did wrong, feel free to add a comment, edit my answer, or write an answer of your own.