How would one be able to clamp a quaternion based on the angle relative to a plane surface (defined by a normal)?

My diagram below provides more of a visual explanation of exactly what I am talking about.

The quaternions are clamped based on the angle between the planes surface, and the quaternion's forward vector (i.e. q * (0, 0, 1)). The quaternions are global quaternions. The plane normal (vector) N defines the upwards direction of the plane surface. Angles that go opposite of the normal vector is negative, whereas in the direction is positive. With that in mind, the quaternion's angle cannot be greater than MAX degrees and less than MIN degrees. These bounds are visualized by the cones (MIN is -90˚ in the diagram, so the cone is infinitely thin).

Clamp Quaternion relative to plane normal


My thought is that this algorithm would have to work in regards to the quaternions delta (unless the quaternion is in the MIN/MAX zones, which I will talk about later). If the change (delta) of the quaternion results in the quaternion entering, or passing, the bounds, the change is capped (clamped) so the final quaternion never has a resulting angle that dissatisfy the bounds.

Clamping the delta could be achieved by determining the beginning and end angles, and obtaining a ratio to be used as the t value (the interpolator) of a Slerp function, so the final quaternion's angle is that of the bound. The issue I see with this is how the roll would be manipulated due to the Slerp. Would the intended, final roll of the quaternion be lost due to this Slerping?

This idea of change seems more important in the example of if the quaternion was to go from -89˚ to -91˚. The final quaternion should be snapped back to -90˚ (if the bound was -90), instead of considering the new quaternion as -89˚ but from the other side. Without this proper interpretation of the quaternion's angle, the quaternion could rotate around constantly and consistently due to the angle never being considered less than -90˚ and greater than 90˚.

Edge Case:

If the quaternion was to start inside the MIN/MAX zones (the cones), the quaternion would not have a change to define as the direction that the quaternion needs to move to get back to legal territory/angles (see angle a3 in diagram). My thoughts is that a fallback plane normal that is orthogonal to plane normal N would have to be used, so the quaternion has a plane to rotate around.

Use Case:

Orbiting camera with user or author definable pitch angle clamping, and a varying change in "up". Example of varying change of "up": Mario Kart 8 Deluxe.

  • \$\begingroup\$ It's unclear to me what angle you're referring to as "the angle" regarding your quaternion. Is it the angle formed between the plane and q * (0, 0, 1)? Or the angle between the plane and q * (0, 1, 0), or something else? \$\endgroup\$
    – DMGregory
    Commented Jun 22, 2020 at 21:47
  • \$\begingroup\$ @DMGregory Formed between the plane and the forward direction of the quaternion. So in most game engines (Unity) that'd be q * (0, 0, 1). Thank you, I missed that clarification. I edited my OP to contain this info. \$\endgroup\$ Commented Jun 23, 2020 at 2:24

1 Answer 1


One idea is that you transform to a coordinate system where up is actually Vector3.up then do whatever clamping you need via standard Euler angles, and then transform it back.

So I think something like this (this is the version in Unity C#, but take it as pseudocode) would work:

    public Quaternion GeneralEulerClamp(Vector3 targetNormal, Quaternion inputrotation) {
        Vector3 xyz = Vector3.Cross(Vector3.up, targetNormal).normalized * Mathf.Sqrt((1 - Vector3.Dot(Vector3.up, targetNormal)) / 2);
        Quaternion coordchange = new Quaternion(xyz.x, xyz.y, xyz.z, Mathf.Sqrt(1 - xyz.sqrMagnitude));
        Vector3 EA = (coordchange * inputrotation).eulerAngles;//Note that quaternion multiplcation is not commutative!
        EA = new Vector3(Mathf.Clamp(EA.x,-90,70), EA.y, EA.z);//set EA.z to zero if you don't want roll
        Quaternion result = new Quaternion();
        result.eulerAngles = EA;
        result = Quaternion.Inverse(coordchange) * result;//It's possible that the first application of coordchange is the inverse and this one isn't
        return result;

targetNormal here needs to be normalized.

So here, coordchange is a rotation rotates targetNormal to Vector3.up. We apply that rotation to the input (by multiplying quaternions), then find the Euler angles. Then we do some clamp operations on the Euler angles and turn in back into a quaternion. Then we do the inverse of the coordchange operation to move Vector3.up back to targetNormal.

Not 100% sure if I worked out my trig correctly since I don't have a way to test this (most likely error is that I got the sign of xyz wrong), but the general concept can work like this.

  • \$\begingroup\$ How exactly did you derive the coordinate change quaternion? After ensuring every possible combination for quaternion multiplication, and changing where coordchange is inverted, I deduced it was the coordchange quaternion causing issues. I could not get the algorithm working for any normals than orthogonal to up and that lay on the ZY plane (normal pointing to X axis does not work). Currently the math behind coordchange just looks like magic to me! :D \$\endgroup\$ Commented Jun 24, 2020 at 15:53
  • \$\begingroup\$ Also two side comments: 1) I've been struggling for months to figure out the rotation order Unity uses to creates Euler Angles, along with the exact code. For those not using Unity, their engine's Euler Angles may be different. This stack exchange post finally has proved to me to convert properly to and from Unity Euler. Though, the angle range of these functions outputs [0, 360], instead of [-180, 180] needed, so EA = ((EA + 180) % 360) - 180; after conversion then EA = EA % 360 after clamp (but latter is not needed due to trig). \$\endgroup\$ Commented Jun 24, 2020 at 16:04
  • \$\begingroup\$ 2) Unity's quaternion multiplication order is opposite that of most game engines. Unity rotates a by b whereas most do b by a. Unity seems to know this, as their new mathematics package does b by a. \$\endgroup\$ Commented Jun 24, 2020 at 16:05
  • \$\begingroup\$ @TarasPalczynski Uh, looks like I messed up, haha. So I was thinking the cross product of two unit vectors gives a vector P with length sin(theta) (theta is the angle between the vectors) perpendicular to both. I want a rotation by theta along P, so quaternion would have w=cos(theta/2) and (x,y,z) with magnitude equal to sin(theta/2) and direction the same as P. So I just did some trig, using that dot product is just the cosine of the angle between the vectors. \$\endgroup\$
    – qwyxivi
    Commented Jun 24, 2020 at 16:54
  • \$\begingroup\$ In hindsight, maybe I should have just answered with the math and not code. \$\endgroup\$
    – qwyxivi
    Commented Jun 24, 2020 at 16:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .