Unity - Proper rotation of object from 3 points in space

Question

I have a rectangular object (for now), with children placed partially in it in its forward, up, and right sides (to make rotations easier to track).

I am trying to get the object to rotate according to 3 points in space. One is the origin (0,0,0), one is in the forward direction (0,0,1), and one is in the right direction (1,0,0).

The origin and forward direction are how I define the origin and direction, so these coordinates are local. I assume for now that the right direction is correct for now.

My simple solution was to set transform.forward = (forward-origin); and transform.right = (right-origin); (where the points are given in global coordinates).

My problem with this is that the rectangular object won't roll around its forward direction. I'm not great with geometry, but I can't get past the thought that setting right ought to solve that problem.

I was, however, not surprised to see that trying to set transform.up as the cross-product of those two made things worse and the object was no longer facing in the forward direction.

So,

1. Why doesn't the combination of forward and right account for this rolling?
2. What if right weren't exactly right? I'm assuming simply normalizing it against forward would do it (and with unity, I could also get up out of it with one function call), but I'm not certain.

Answer 1

When you set one direction vector at a time, Unity doesn't know you're about to set another one. The semantics of the language make each assignment a separate operation, and it needs to give you a complete new orientation after each one, not a partial transformation-in-progress.

So if you set transform.forward = foo, Unity has 2 of 3 degrees of freedom specified, and it has to guess what you wanted for the third. Sometimes it will guess wrong.

Then if you set transform.right = bar, Unity again has 2 of 3 degrees of freedom specified, and it has to guess again. Sometimes it will guess wrong.

It doesn't know the two operations are connected - for all it can tell, you changed your mind after the first assignment, and now want a completely different, unrelated orientation from the second assignment. (If it didn't make this assumption, transform operations would have a "memory" you'd have to manually clear between uses, which would be clunky)

If you want both forward and right to point in particular directions, you need to express that as a single transformation, not as two separate steps. Here's one way to do it:

Vector3 forwardDirection = forwardPosition - origin;
Vector3 rightDirection = rightPosition - origin;
Vector3 upDirection = Vector3.Cross(forwardDirection, rightDirection);

Quaternion orientation = Quaternion.LookRotation(forwardDirection, upDirection);

transform.rotation = orientation;

This will give you an orientation whose forward vector points exactly along your chosen forward direction, and whose right vector points as close as it can along your chosen right direction while still obeying the first constraint (since we have only 1 degree of freedom left).