Quaternion Look At with Up vector

Question

I have implemented a "look at" method for my screen elements and it almost works how I want it to work.

The ScreenElement class uses a R3 Vector and a Quaternion to determine the position and orientation. As a convenience function I implemented a "look at" method. I am not the biggest expert on linear algebra, so my solution was an adaption of stuff I found on forums; but it was the only solution I understood.

The current approach is as follows:

void SceneElement::look_at(const mx::Vector3f& target, const mx::Vector3f& up)
{
    mx::Vector3f forward_l = mx::normalize(target - position);
    mx::Vector3f forward_w(1, 0, 0);
    mx::Vector3f axis  = forward_l % forward_w;
    float        angle = mx::rad_to_deg(acos(forward_l * forward_w));

    mx::Vector3f third = axis % forward_w;
    if (third * forward_l < 0)
    {
        angle = - angle;
    }
    orientation = mx::axis_angle_to_quaternion(angle, axis);
}

As you can see the up vector is not used and as a result the orientation is so far correct, that it looks as the target, but the up orientation is all over the place.

I have tried different approached to integrate the up vector. The "best" (i.e. it still mostly looked at the target) was to make the same approach with a orthonormalized up vector.

Just as a note, the up vector is expected to work like gluLookAt, that is used as a baseline but it needs to be projected into the local coordinate system, this I did as:

mx::Vector3f right = mx::normalise(forward % up);
mx::Vector3f up_n  = mx::normalise(right % forward);

Just to complete the info, the math lib I use is my own mathex and the basterised modulo (%) operator is the R3 cross product.

Answer 1

I solved it. I was on the right track... The final solution is:

void SceneElement::look_at(const mx::Vector3f& target, const mx::Vector3f& up)
{
    mx::Vector3f forward_l = mx::normalize(target - position);
    mx::Vector3f forward_w(1, 0, 0);
    mx::Vector3f axis  = forward_l % forward_w;
    float        angle = mx::rad_to_deg(acos(forward_l * forward_w));

    mx::Vector3f third = axis % forward_w;
    if (third * forward_l < 0)
    {
        angle = - angle;
    }
    mx::Quaternionf q1 = mx::axis_angle_to_quaternion(angle, axis);

    mx::Vector3f up_l  = mx::transform(q1, mx::normalize(up));
    mx::Vector3f right = mx::normalize(forward_l % up);
    mx::Vector3f up_w  = mx::normalize(right % forward_l);

    mx::Vector3f axis2  = up_l % up_w;
    float        angle2 = mx::rad_to_deg(acos(up_l * up_w));

    mx::Quaternionf q2 = mx::axis_angle_to_quaternion(angle2, axis2);

    orientation = q2 * q1;
}

The only open issue I have is I am not sure in which cases the commented out code is necessary.