Quaternion precision problems

Each entity and camera have a Transformation object that holds position info as vec3, rotation info as quat and scale info as vec3.

When I need a matrix (that being either model or view matrix), I construct the matrix out of given data. That gives me the freedom to change any of transformation info without having to worry about order of change.

Now, I'm trying to implement it in a fly-camera. The two constraints are pitch that has to be limited from -90° to 90° and roll which should not occur.

While rotating the camera along X and Y axis, a sort of drift happens along the Z axis. In order to combat that, after all changes to rotation are applied, I get the angle of rotation around the Z axis and then rotate it by -angle, effectively resetting the Z-rotation to 0.

That all works nicely in theory, but not so much in practice.

You see, when I go over ~55ish° both in the positive or negative Y-rotation, the camera starts freaking out and fluctuating. It seems that every frame it switches from some negative z rotation, to positive z rotation of the same value. It looks like this (moving camera to the left, negative Y-rotation):

Of course, since the object is only in the one place at one time, I had to edit the screenshot in paint to give you a better idea of what's going on. It really does look like there are two objects at the same time, both flickering as crazy.

Also note that if I were to move camera further to the left, the Z-rotation offset would increase, moving both objects further from the horizontal plane. At this point, the Y-rotation would also start to fluctuate ever so slightly. You couldn't see it on screen, but printing out angle each frame would show that it's fluctuating at about 3rd digit after the dot (0.00####).

Once I get far enough, the camera would just give up and just show the object rapidly changing it's Z-rotation with no apparent pattern.

My guess is that quaternion rotation is imprecise for some reason. You can tell by the fact that my strict Z-rotation starts to influence the Y-rotation after the Z-drift gets severe enough.

Here's a snippet of code, for what it's worth:

transformation->rotateAroundY(angleY);

GLfloat roll = transformation->getRoll();
transformation->rotateAroundZ(-roll);


angleY is delta Y of mouse position between this and previous frame. rotateAround#(angle) calls this code:

rotation *= glm::tquat<GLfloat>(glm::angleAxis(glm::radians(angle), axis));


where rotation is the quaternion, while axis is tvec3<GLfloat> representing the axis around which the rotation takes place.

How do I combat this? How can I consistently reverse unintentional Z-drift without making the camera freak out?

• This doesn't look like a precision problem to me. Composing local pitch and yaw rotations frame to frame naturally introduces roll because these operations aren't globally orthogonal — rotations wrap around unlike Euclidean translations. It looks like your math for counter-rolling is applying the rotation about the world z axis, after your rotation. Instead, you want to apply the counter-roll along the local z axis, before rotation. It looks almost right at shallow angles because the axes are near-parallel, but it's the wrong correction to apply. – DMGregory Oct 29 '16 at 18:47
• The quaternion starts with 0 rotation around the Z-axis. If I apply correction before frame X-rotation and Y-rotation, I change nothing. Z-rotation is still 0. Z-drift appears only after the X/Y rotation. That drift would be reset to 0 the next frame, but not before the renderer passes it to the shader. I tried swapping the rotateAroundZ with rotateAroundY and I noticed no change. I'd like to emphasize that Z-rotation jumps from -z to z each frame. Could you write a simple pseudo code to demonstrate your idea, I might be missing the point completely? – Karlovsky120 Oct 29 '16 at 19:16
• When I say apply the z correction before your other rotation I mean this: imagine myRotation.roll = 30. Then I build correctingRotation = Roll(-30). Then I apply myRotation to it: myCorrectedRotation = correctingRotation.RotateBy(myRotation). What you're doing is the opposite, applying the counter-roll at the end of the transform chain instead of the beginning – DMGregory Oct 29 '16 at 19:22
• Hmmm, but I have to myRotation first to be able to measure the drift. This way, I'd have to do all normal rotations, measure the correction needed, undo all rotations, apply the correction and then rotate the all again, no? – Karlovsky120 Oct 29 '16 at 19:28
• Not really. I showed it that way to make the order explicit, but with matrices or quaternions it's all about which side you multiply on. Multiplying your correction on one side puts it at the beginning of the transform order, and multiplying on the other side applies it at the end. – DMGregory Oct 29 '16 at 19:31