Quaternions distorting object in Metal

Question

I've been trying to get rotations with quaternions to work for a while now, and I feel I am very close to it actually working but I just can't get it to work.

I do not have a view matrix yet and am just working in 2D space to make this slightly easier to set at the moment. So I'm only rotating around the Z axis.

My model matrix is calculated like this

var modelMatrix: matrix_float4x4 {
    var modelMatrix = matrix_identity_float4x4
        
    modelMatrix.translate(_position)
        
    modelMatrix.rotate(_orientation)
        
    modelMatrix.scale(_scale)
        
    return matrix_multiply(matrix_identity_float4x4, modelMatrix)
}

_orientation is defined as

private var _orientation: quaternion = quaternion(angle: toRadians(0), axis: float3(0, 0, 1))

quaternion is just a typedef of simd_quatf

my rotate function is

mutating func rotate(_ quat: quaternion) {
    let quatMatrix = matrix_float4x4(simd_normalize(quat))
        
    self = matrix_multiply(self, quatMatrix)
}

I also tried to do it with the .act method

mutating func rotate(_ quat: quaternion, _ vec: float3) {
    let rotatedVec = quat.act(vec)
        
    self.translate(rotatedVec)
}

This did the seemingly same thing as the first rotate function.

When I initialize _orientation with a rotation of 0, the program acts as normal but when I initialize it with 45, it warps the rectangle:

and when I do it with 90, it produces the same result as with 0, which shouldn't happen. I'm assuming the problem lies somewhere in the way I'm applying the rotation to the object, but I have no clue what it is.

Edit:

Also A thought, maybe it is because of the way I am applying the modelMatrix to the shader positions. I'm pretty sure this is the right way but it might be slightly wrong.

vertex RastData vertexShader(const VertexIn vIn [[stage_in]],
                             constant ModelMat &modelMatrix [[buffer(1)]]) {
    RastData rd;
    
    rd.position = modelMatrix.modelMatrix * float4(vIn.position, 1.0);
    rd.color = vIn.color;
    
    return rd;
}

Edit 2: I Have tried to do it with the math described in this answer https://gamedev.stackexchange.com/a/188776/157187 but that did nothing

mutating func rotate(_ angle: Float, _ axis: float3, _ pQuat: quaternion) {
        let newQ = quaternion(vector: float4(axis, cos(angle / 2)))
        
        let newPQ = newQ * pQuat * newQ.conjugate
        
        let newVec = float3(newPQ.vector.x, newPQ.vector.y, newPQ.vector.z)
        
        var mat = matrix_identity_float4x4
        
        mat.columns = (
            float4(1, 0, 0, 0),
            float4(0, 1, 0, 0),
            float4(0, 0, 1, 0),
            float4(newVec.x, newVec.y, newVec.z, 1)
        )
        
        self = matrix_multiply(self, mat)
}

Edit 3: Alright I think the problem lies in my lack of a view and projection matrix, so I have defined some quick ones, not really implemented a camera system, just having the functionality of the view and projection matracies.

I have a function that makes an orthographic projection:

func makeOrthographicMatrix(tooManyArgsLol) { 
return float4x4(
        float4(2 / (right - left), 0, 0, 0),
        float4(0, 2 / (top - bottom), 0, 0),
        float4(0, 0, 1 / (far - near),   0),
        float4((left + right) / (left - right), (top + bottom) / (bottom - top), near / (near - far), 1)
    )
}

This uses the Column constructor so it looks weird but it's correct.

And a view matrix that should be -3 on the Z axis:

var viewMatrix: float4x4 {
        var viewMat = matrix_identity_float4x4
        
        viewMat.translate(float3(0, 0, -3))
        
        return viewMat
}

but now all I get is the clearColor I set.

In my vertex shader I do the multiplication like this

cameraMats.projectionMatrix * cameraMats.viewMatrix * modelMatrix.modelMatrix * float4(vIn.position, 1.0);

When I debug using the Metal debugger, it shows that all the data is going through to the shader correctly, so it's definitely some sort of math issue, I just don't really know where to look now.

Answer 1

Alright so the main problem was actually that I had to projection matrix that changed depending on the aspect ratio of the window that was being rendered to. So with that out of the way I actually did end up getting quaternion rotations to work:

mutating func quatRotate(_ angle: Float, _ axis: float3) {
    var q = quaternion(angle: angle.toRadians, axis: axis)
        
    let qMat = float4x4(q.normalized)
        
    self = qMat * self
}

And use this before I translate like so:

var viewMatrix: float4x4 {
    var vm = matrix_identity_float4x4
        
    vm.quatRotate(_orientation.x * 50, XAXIS)
    vm.quatRotate(_orientation.y * 50, YAXIS)
    vm.quatRotate(_orientation.z * 50, ZAXIS)
        
    vm.translate(_position)
        
    return vm.inverse
}

The * 50 is because just using _orientation.angle was like a 1 pixel per second rotation speed lol.

Thank you to everyone who commented on this despite it being hot mess in the context of me actually knowing what was wrong! @DMGregory the goat I love you.

Answer 2

I am not familiar with Metal's API so I can't be 100% sure on the usage correctness but mathematically what must happen is the following:

1. Create a model matrix by applying scaling, rotation then translation, in that order (you are doing the reverse)
2. Apply the resulting matrix to each point of the model (that is what the vertex shader does)

Code-wise this would look like:

var modelMatrix: matrix_float4x4 {
    var modelMatrix = matrix_identity_float4x4
    modelMatrix.scale(_scale)
    modelMatrix.rotate(_orientation)
    modelMatrix.translate(_position)
    return modelMatrix
}

Also the rotation method might need to change to

mutating func rotate(_ quat: quaternion) {
    let quatMatrix = matrix_float4x4(simd_normalize(quat))
    self = matrix_multiply(quatMatrix, self) // the rotation matrix must be multiplied from the LEFT
}

For the rotation, you must make sure that the quatMatrix value ends up being $$C=\begin{pmatrix} 0 & -1 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{pmatrix}.$$ or something very close to it (i.e. the 1x1 and 2x2 cells should be very small values). This is for clockwise 90 degree rotation.

As mentioned, I am not familiar with the API at all so the fixes I've suggested might be incorrect or not needed but that's the theory at least.

The vertex shader looks correct to me, again with assumptions on Metal's shader language.

Finally, remember that since vertex shaders output normalized device coordinates (that is, all coordinates range from \$[-1, 1]\$) you have to compensate for the viewport aspect ratio distortion. What this means is that if you have a square in the vertex shader for example with the top left corner at \$(-200,-200)\$ and bottom right at \$(200,200)\$, these must not be mapped to something like \$(-0.2, -0.2)\$ and \$(0.2, 0.2)\$. Instead the x coordinate must be divided by the viewport's aspect ratio \$(-0.2 / aspectRatio, -0.2)\$ in order for a square to be rendered on the screen. This can be easily done by applying an orthographic projection matrix after the model matrix (view this page https://en.wikipedia.org/wiki/Orthographic_projection on how to create it).