Finding pitch/yaw values from lookat vector

Question

I have a camera class that contains a position, a pitch, and a yaw (the pitch and the yaw correspond to y and x rotation values). I want to implement a lookAt function that takes a vector and the function changes the pitch and yaw so it's looking at that point in space. However, I cannot seem to find the math equation to do it (various ones online, none seem to work). How would I be able to implement it?

Here's an example of what I mean:

struct Camera
{
    Vec3 position;
    float pitch, yaw;

    void lookAt(Vec3 center)
    {
        //????
    }

    mat4x4 toTransform() //How I convert the values to a matrix (if this is relevant)
    {
        return CreateLookAtMatrix
            position,
            position + Vec3(cos(pitch)*sin(yaw), sin(pitch), cos(pitch)*cos(yaw)),
            Vec3(0.0f, 1.0f, 0.0f)
            );
    }
}

Answer 1

You've already got the answer in your matrix function:

lookAt = position + Vec3(cos(pitch)*sin(yaw), sin(pitch), cos(pitch)*cos(yaw))

You just need to do a little algebra to get the yaw and pitch variables over to the left.

To start with the simplest expression, the y-axis gives:

lookAt.y = position.y + sin(pitch), which we can rearrange to...
sin(pitch) = lookAt.y - position.y
pitch = asin(lookAt.y - position.y)

Where asin is arcsine, the inverse of sine.

Note that this assumes lookAt - position is a unit vector (length 1) which in your matrix expression is true by construction. If center can be anywhere then you'll want to start by constructing the unit vector you need:

void lookAt(Vec3 center)
{
   Vec3 direction = (center - position).normalized;
   pitch = asin(direction.y);
   yaw = atan2(direction.x, direction.z);
}

I skipped ahead a bit on yaw there because we have both k*sin(yaw) and k*cos(yaw) for some real k, so we can just drop that into the atan2 convenience function and let it sort out all the cases for us. ;)