World rotation independent mouselook

Preface: I am terrible at using quaternions right.

I wanted to create a world rotation independent mouselook script. That is, no matter the 'down' for the camera, the mouselook should feel natural to the player in relation to what is displayed on screen.

I now have the following script running on update in my camera:

        Quaternion worldRot = Quaternion.LookRotation(Vector3.forward, Vector3.left);

xRot += -rotSpeed * Input.GetAxis("Mouse Y");
yRot += rotSpeed * Input.GetAxis("Mouse X");
transform.rotation =
Quaternion.AngleAxis(yRot, worldRot * Vector3.up) *
Quaternion.AngleAxis(xRot, worldRot * Vector3.right) *
worldRot;


Unsurprisingly, it tumbles all over the place. I think the issue is in basing the rotation angles on the worldRot (World rotation as seen by the camera), while then also rotating the whole by worldRot. I cannot get my head around how else to go about it, though...

My initial alternative was to use

        transform.rotation =
Quaternion.AngleAxis(yRot, Vector3.up) *
Quaternion.AngleAxis(xRot, Vector3.right) *
worldRot;


But this anchors the mouse's movements to the world rather than the camera, which as described above isn't desired either.

Based on this (or, perhaps, in a completely different way), how do I make a mouselook system with an arbitrary down vector?

For a typical FPS camera that can't look straight up/down (to stay away from the gimbal lock point where the view just spins), I'd use Quaternion.Euler to construct it from your yaw and pitch variables.

We'll measure our yaw and pitch relative to the current world orientation, so all we have to do is multiply this by the world orientation quaternion to get our net orientation:

Quaternion GetUpdatedCameraRotation(Quaternion worldOrientation) {

yawDegrees = (yawDegrees + rotationSpeed  * Input.GetAxis("Mouse X")) % 360.0f;
pitchDegrees = Mathf.Clamp(pitchDegrees - rotationSpeed * Input.GetAxis("Mouse Y"),
-maxPitchDegrees, maxPitchDegrees);

return worldOrientation * Quaternion.Euler(pitchDegrees, yawDegrees, 0f);
}


When the world orientation changes, we'll need to update our yaw and pitch to preserve the direction we're looking, under the new frame of reference:

void UpdateWorldOrientation(Quaternion newWorldOrientation) {

Vector3 forward = Quaternion.Inverse(newWorldOrientation) * transform.forward;

yawDegrees = Mathf.Atan2(forward.x, forward.z) * Mathf.Rad2Deg;
}


Note that as-written, "up" on the screen will always point to the current world up - you'll never be twisted sideways. If your world orientation changes suddenly, you'll likely want to blend to the new camera orientation over several frames (eg. using Quaternion.RotateTowards), so the player sees the camera twist to orient to the new up/down, rather than suddenly popping into a new orientation.

In hindsight, I was way overcomplicating this.

The usual y rotation should be done around transform.up rather than vector3.up. The usual x rotation should be done around transform.right rather than Vector3.right.

No explicit quaternion multiplications needed.

It was ultimately quite simple. Thanks for all the pointers.

As for the resulting code; I first made the camera actually a child of the object that actually gets rotated. Because of that, the camera's rotation becomes local to that rotation. Therefore, the actual mouselook script is no more than

    Vector2 look = input.Player.Look.ReadValue<Vector2>();
look *= Time.deltaTime;
yLook += look.y * LookSensitivity;
yLook = Mathf.Clamp(yLook, -85, 85);
transform.Rotate(Vector3.up * (look.x * LookSensitivity));
camera.transform.localRotation = Quaternion.Euler(-yLook, 0, 0);


Mind you that this is run by the parent with reference to the child. It is holding on to a yLook field because we rotate the camera by it, but don't want it to influence the camera's movement. The parent object runs the following code on itself.

transform.rotation = Quaternion.LookRotation(World.transform.up, World.transform.forward);


I found that separating the code out like this allowed me to do more complex camera movements elsewhere in the script. At no point does the camera itself really need to be concerned with what 'up' is.

• Careful, using local axes for both pitch and yaw can start to introduce roll into your view through certain combinations, depending on how you implement this. Jul 11, 2020 at 17:29
• @DMGregory I see how that could happen if they were applied at once sequentially, but via a lookrotation I've had no such issues. And I've sent my camera through a rigorous testing course. Jul 11, 2020 at 17:48
• Yep, LookRotation keeps the up vector aligned with world up or the second argument, so that's a good solution to the problem here. Want to share the LookRotation code you're using so other folks can avoid that trap as you have? Jul 11, 2020 at 17:53
• @DMGregory I can't get to my codebase right now but I'll do so tonight. It's practically a one-liner, but let's get it right. Jul 11, 2020 at 18:35
• Ah, what's saving you here isn't the LookRotation, it's the parenting. Because you're applying the pitch and yaw on two different objects, they don't get the axis cross-contamination effects described in the thread I linked above. That's a solid solution too. If this worked for you, feel free to mark it Accepted - I don't mind losing the checkmark. ;) Jul 12, 2020 at 11:50

I assume that since you said "no matter the 'down' for the camera", you want the rotation to be relative to the camera and you are not actually rotating "the world".

Then one reason your original script was not working is because

Quaternion worldRot = Quaternion.LookRotation(Vector3.forward, Vector3.left);

should have been

Quaternion worldRot = Quaternion.LookRotation(transform.forward, transform.up);

which is the same as just doing Quaternion worldRot = transform.rotation;. Then your second snippet would have worked.

• No, this instance of worldRot was an arbitrary example. It should work for any value of that. Apr 2, 2020 at 12:17