# How to clamp a direction vector within a direction pyramid?

In 2D you can specific an angle range such as [-45 to +45] and clamp a 2D direction vector within that angle range. I want to have something similar for 3D. I want to specify the allowed directional range as a center directional axis with a local y-axis range and a local x-axis range and clamp an arbitrary 3D directional vector within that range. How do I do that?

For example, I want to specify a angle range as axis (1,0,0) with [-30 to 30] as the local y-axis angle angle and [-10 to 10] as the local x-axis range. So that something like (0.95,0.1,0.1).normalized() would not be changed but (-1,0,0) would be clamped to an appropriate vector.

• Are the angle ranges always symmetrical? Is there a minimum/maximum on the ranges? What is the desired result for (-1,0,0)? – Jon Mar 14 '16 at 6:51
• a cone is good or it has to be a pyramid ? the cone is much easier, you just have a scalar product to make. the pyramid, hm... a 4x4 matrix ? it's equivalent to a view frustum so all projection math should apply – v.oddou Mar 17 '16 at 1:18

Assuming your pyramid is defined by a direction vector, an up vector (for the general case, that the pyramid is rotated, it probably defaults to (0, 1, 0)) and two angles maxPitch and maxYaw, I came up with the following solution (in c++, but easily translateable to other languages):

class ViewingPyramid {
private:
Vector3 direction, up, right;
float maxYaw, maxPitch;

public:
ViewingPyramid(Vector3 dir, Vector3 up, float yawmax, float pitchmax) :
direction(Vector3::Normalize(dir)),
up(Vector3::Cross(direction, Vector3::Normalize(Vector3::Cross(up, direction)))),//Getting a rectangular, orthogonal up vector
right(Vector3::Cross(direction, this->up))
{}

Vector3 Clamp(Vector3 dir) {
Vector3 abc = getSolutions(right, up, direction);

Vector3 xz = Vector3::Normalize(dir - abc.Y*up);
float yaw = Math::Clamp(Vector3::Angle(direction, xz), -maxYaw, maxYaw);

Vector3 yz = Vector3::Normalize(dir - abc.X*right);
float pitch = Math::Clamp(Vector3::Angle(direction, yz), - maxPitch, maxPitch);

Vector3 res = direction;
Quaternion qYaw = Quaternion::CreateFromAngleAxis(up, yaw);
Quaternion qPitch = Quaternion::CreateFromAngleAxis(right, pitch);
res *= qYaw;
res *= qPitch;

return res;
}
};


As you can see, it relies on some prebuilt math functions, which should mostly be easy to implement. I did not test it, because I don't have getSolutions implemented in my simple math library, but I'm pretty confident this works as intended. You can use whatever library you want (For example Eigen in C++) for math functions, or implement your own. The most complicated part is the getSolutions function, which gets the solutions for the equation dir = a*right + b*up + c*direction.

I hope this helps.

• OP here, but I lost the ability to access the account and I cant comment. Can you elaborate on the implementation of getSolution? I am not clear what it is suppose to do. In any case, I solved my problem by implementing a method to calculate the signed angle between two vectors in a given reference plane. With that, I am able to calculate the local pitch yaw and clamp them to appropriate values. – user80667 Mar 17 '16 at 0:26
• @user80667 getSolutions essentially splits the given directions vector into its local forward-, up- and right-components (It does so by getting the solution to equation I mentioned in the answer). With this you are able to calculate the pitch and yaw angles (like you did) and clamp them. – LukeG Mar 17 '16 at 8:50