World rotation independent mouselook

Question

Preface: I am terrible at using quaternions right.

I wanted to create a world rotation independent mouselook script. That is, no matter the 'down' for the camera, the mouselook should feel natural to the player in relation to what is displayed on screen.

I now have the following script running on update in my camera:

        Quaternion worldRot = Quaternion.LookRotation(Vector3.forward, Vector3.left);

    xRot += -rotSpeed * Input.GetAxis("Mouse Y");
    yRot += rotSpeed * Input.GetAxis("Mouse X");
    transform.rotation =
        Quaternion.AngleAxis(yRot, worldRot * Vector3.up) *
        Quaternion.AngleAxis(xRot, worldRot * Vector3.right) *
        worldRot;

Unsurprisingly, it tumbles all over the place. I think the issue is in basing the rotation angles on the worldRot (World rotation as seen by the camera), while then also rotating the whole by worldRot. I cannot get my head around how else to go about it, though...

My initial alternative was to use

        transform.rotation =
        Quaternion.AngleAxis(yRot, Vector3.up) *
        Quaternion.AngleAxis(xRot, Vector3.right) *
        worldRot;

But this anchors the mouse's movements to the world rather than the camera, which as described above isn't desired either.

Based on this (or, perhaps, in a completely different way), how do I make a mouselook system with an arbitrary down vector?

Answer 1

For a typical FPS camera that can't look straight up/down (to stay away from the gimbal lock point where the view just spins), I'd use Quaternion.Euler to construct it from your yaw and pitch variables.

We'll measure our yaw and pitch relative to the current world orientation, so all we have to do is multiply this by the world orientation quaternion to get our net orientation:

Quaternion GetUpdatedCameraRotation(Quaternion worldOrientation) {

    yawDegrees = (yawDegrees + rotationSpeed  * Input.GetAxis("Mouse X")) % 360.0f;
    pitchDegrees = Mathf.Clamp(pitchDegrees - rotationSpeed * Input.GetAxis("Mouse Y"),
                  -maxPitchDegrees, maxPitchDegrees);

    return worldOrientation * Quaternion.Euler(pitchDegrees, yawDegrees, 0f);
}

When the world orientation changes, we'll need to update our yaw and pitch to preserve the direction we're looking, under the new frame of reference:

void UpdateWorldOrientation(Quaternion newWorldOrientation) {

    Vector3 forward = Quaternion.Inverse(newWorldOrientation) * transform.forward;

    pitchDegrees = Mathf.Asin(-forward.y) * Mathf.Rad2Deg;

    yawDegrees = Mathf.Atan2(forward.x, forward.z) * Mathf.Rad2Deg;
}

Note that as-written, "up" on the screen will always point to the current world up - you'll never be twisted sideways. If your world orientation changes suddenly, you'll likely want to blend to the new camera orientation over several frames (eg. using Quaternion.RotateTowards), so the player sees the camera twist to orient to the new up/down, rather than suddenly popping into a new orientation.

Answer 2

In hindsight, I was way overcomplicating this.

The usual y rotation should be done around transform.up rather than vector3.up. The usual x rotation should be done around transform.right rather than Vector3.right.

No explicit quaternion multiplications needed.

It was ultimately quite simple. Thanks for all the pointers.

As for the resulting code; I first made the camera actually a child of the object that actually gets rotated. Because of that, the camera's rotation becomes local to that rotation. Therefore, the actual mouselook script is no more than

    Vector2 look = input.Player.Look.ReadValue<Vector2>();
    look *= Time.deltaTime;
    yLook += look.y * LookSensitivity;
    yLook = Mathf.Clamp(yLook, -85, 85);
    transform.Rotate(Vector3.up * (look.x * LookSensitivity));
    camera.transform.localRotation = Quaternion.Euler(-yLook, 0, 0);

Mind you that this is run by the parent with reference to the child. It is holding on to a yLook field because we rotate the camera by it, but don't want it to influence the camera's movement. The parent object runs the following code on itself.

transform.rotation = Quaternion.LookRotation(World.transform.up, World.transform.forward);

I found that separating the code out like this allowed me to do more complex camera movements elsewhere in the script. At no point does the camera itself really need to be concerned with what 'up' is.

Answer 3

I assume that since you said "no matter the 'down' for the camera", you want the rotation to be relative to the camera and you are not actually rotating "the world".

Then one reason your original script was not working is because

Quaternion worldRot = Quaternion.LookRotation(Vector3.forward, Vector3.left);

should have been

Quaternion worldRot = Quaternion.LookRotation(transform.forward, transform.up);

which is the same as just doing Quaternion worldRot = transform.rotation;. Then your second snippet would have worked.