Limit rotation/direction of two axes with seperate values (Vector3)

Question

Issue

I'm trying to create a vector pointing in a direction towards a target, and then using that vector rotate a child transform to face the target while limiting the rotation on the X and Y axis, with seperate min/max values per axis.

Current Code

Vector3 targetVector = Camera.main.transform - transform.parent.position;
Vector3 rotateVector = Vector3.RotateTowards (transform.parent.forward, targetVector, maxRotation * Mathf.Deg2Rad, 0f);

transform.rotation = Quaternion.LookRotation (rotateVector, transform.parent.up);

The above code is closest I've come to getting the child transform to face the target while limiting how much it can rotate, what it doesn't allow me to do though is set different rotation limits on each axis.

Theory

Can anyone confirm it's possible to use the forward and up vector of the parent transform to create some form of zeroed rotation reference, and then using the direction vector towards the target to create another vector which is clamped to not exceed some value of degrees on either axis - and then finally use that vector to set the rotation of the child object?

For example, not exceeding negative ten or positive forty five degrees on the Y axis, while also not exceeding negative forty five or positive seventy on the X axis away from the parent transform forward vector.

Images

In the hopes of better explaining exactly what I'm trying to do I've created a couple of GIF - represented in them are:

• The pink cross is the target transform
• The orange line is the direction to the target transform vector.
• The solid blue line is the forward vector of the child transform.
• The solid green line is the up vector of the child transform.
• The dashed blue line is the forward vector of the parent transform.
• The dashed green line is the up vector of the parent transform.
• The brighter green arc represents the ideal min/max rotation of the target vector.
• The brighter red arc represents the ideal min/max rotation of the target vector.
• The darker green arc represents the current limited rotation of the target vector.
• The darker red arc represents the current limited rotation of the target vector.

In-closing

The crux of the issue is I'm dumb as all hell, and vectors, euler angles and quaternions are the work of the devil so any help would be greatly appreciated, and thank you to anyone who reads this.

Attempted Solution

Using the code as suggested by DMGregory:

Vector3 targetDirection = (transform.parent.InverseTransformPoint (targetPosition) - transform.localPosition).normalized;

float yaw = Mathf.Atan2(targetDirection.x, targetDirection.z) * Mathf.Rad2Deg;
float pitch = Mathf.Asin(targetDirection.y) * Mathf.Rad2Deg;

yaw = Mathf.Clamp(yaw, minYaw, maxYaw);
pitch = Mathf.Clamp(pitch, minPitch, maxPitch};

transform.localRotation = Quaternion.Euler (pitch, yaw, 0);

The pitch angle seems to be reversed, I think it is because the default rotation of the Wrist Bone is one hundred and eighty on the Y axis, while the default rotation of the Hand Bone and Right Hand is zero on the same axis.

Vector3 targetDirection = initialRotation * (transform.parent.InverseTransformPoint (Camera.main.transform.position) - transform.localPosition).normalized;

float yaw = Mathf.Atan2 (targetDirection.x, targetDirection.z) * Mathf.Rad2Deg;
float pitch = Mathf.Asin (targetDirection.y) * Mathf.Rad2Deg;

yaw = Mathf.Clamp (yaw, limitYaw.min, limitYaw.max);
pitch = Mathf.Clamp (pitch, limitPitch.min, limitPitch.max);

transform.localRotation = Quaternion.Euler (pitch, yaw, 0f) * initialRotation;

I've "fixed" the issue by multiplying both the targetDirection and Quaternion.Euler by the initial rotation of the child transform: I'm unsure if this is actually a fix, or just seems to work in the limited amount of testing I have done, do you have any ideas on how to solve this correctly?

---

Thank you very much, I've been trying to do this for weeks (even using code very similar to yours) with zero success until now - I really do appreciate the time you've spent helping me.

Probable Solution

^{Abondon all hope and finally give up - not just on ever getting this working but on life itself.}

Answer 1

Abondon all hope and finally give up - not just on ever getting this working but on life itself

There's no need to get so dramatic. This is a pretty simple application of spherical coordinates.

// Transform the target point into our parent's coordinate space,
// and compute the direction toward it.
Vector3 targetDirection = (
    transform.parent.InverseTransformPoint(
       targetPosition
    ) - transform.localPosition
 ).normalized;

// Convert this local direction into spherical coordinates.
float yaw = Mathf.Atan2(targetDirection.x, targetDirection.z) * Mathf.Rad2Deg;
float pitch = Mathf.Asin(targetDirection.y) * Mathf.Rad2Deg;

// Apply your desired clamping.
yaw = Mathf.Clamp(yaw, minYaw, maxYaw);
pitch = Mathf.Clamp(pitch, minPitch, maxPitch};

// Convert angles into a local orientation quaternion.
transform.localRotation = Quaternion.Euler(pitch, yaw, 0);

This is assuming that you want your min/max specified relative to the parent's orientation — where 0 pitch/yaw points along the parent's forward vector. If you want a different reference coordinate system that can be arranged with a small extra effort.

Answer 2

I think your original solution was correct with one mistake:

Vector3 targetVector = Camera.main.transform - transform.parent.position;
Vector3 rotateVector = Vector3.RotateTowards(transform.parent.forward, targetVector, maxRotation * Mathf.Deg2Rad, 0f);
// maxRotation should be updated every frame to incorporate the already applied rotation value
transform.rotation = Quaternion.LookRotation(rotateVector, transform.parent.up);

I'm guessing this code snippet is called on every Update(), in which case the problem is that you're not properly "updating" the maxRotation value. The maxRotation will clamp the rotation value for the specific call of RotateTowards, this means that for every Update() call, if the rotation would be larger than the maxRotation value it is clamped. However the next call will not take into account the already applied rotation on your object. You are essentially moving the goal post further and further, what you should do is subtract from the maxRotation value, the current rotation of the transform and once rotation reaches its maximum value then maxRotation should be 0.

EDIT: Giving an example in code (not sure if my Unity script is correct) I'd do:

private readonly float maxRotation = 5f;
private float currentMaxRotation = maxRotation;

Update() {
  Vector3 targetVector = Camera.main.transform - transform.parent.position;
  Vector3 rotateVector = Vector3.RotateTowards(transform.parent.forward, targetVector, currentMaxRotation * Mathf.Deg2Rad, 0f);
  float rotatedAmount = Vector3.Angle(transform.parent.forward, rotateVector);
  this.currentMaxRotation -= rotatedAmount; // if we rotated by 2, then in the next Update call we'll have a maximum of 3 now, until we reach 0.
  if(this.currentMaxRotation < 0) this.currentMaxRotation = 0;
  transform.rotation = Quaternion.LookRotation(rotateVector, transform.parent.up);
}

This code obviously needs extra parts for resetting the "currentMaxRotation" to be non 0 again but I wanted to demonstrate the gist.