Extract a rotation matrix given a camera direction (vector) and a up vector for the camera?

Question

I am having some issues constructing a rotation matrix from a camera direction vector.

The Up vector is computed from the camera direction as such:

up = cross(cross(direc,(0,1,0)),direc)

How do i given the up vector and a camera direction vector compute the rotation vector, that tells the camera how it should be rotated such that it points toward my object?

Answer 1

NOTE: Edited because it was likely too verbose (source).

A rotation matrix actually always defines an orthonormal basis. What this means is each column defines one of your original axes in its rotated state. For example, consider a simple rotation matrix around the z-axis (more on rotation matrices here). Let's say we plug in Pi / 2, in other words, we rotate counterclockwise at a right angle.

| cos(Pi/2) -sin(Pi/2) 0 |   | 0 -1  0 |    x-axis: ( 0, 1, 0)
| sin(Pi/2)  cos(Pi/2) 0 | = | 1  0  0 |    y-axis: (-1, 0, 0)
|    0          0      1 |   | 0  0  1 |    z-axis: ( 0, 0, 1)

Which is exactly what you'd expect. The z-axis stays the same. The x-axis rotates 90 degrees, to where the y-axis used to be, and the y-axis rotates 90 degrees further from that, to the negative x-axis.

In camera coördinates, I'm going to assume the following (which is free of choice):

• The x-axis (column 1) points to the right of your camera: the right vector.
• The y-axis (column 2) points upwards from your camera: the up vector.
• The z-axis (column 3) points straight out of your camera: the forward vector.

You already have your forward vector (I'm assuming that's the 'direc' vector you're talking about). However, the way you're calculating your up vector doesn't seem right. If you know the right hand rule, apply it twice for each of your cross products, and you'll end up with some vector pointing towards the negative y-axis, which is probably not what you want.

There's actually a few ways to calculate your remaining two vectors, but it's not quite clear what you're going for, so I'll give you the most straightforward (but least flexible) approach. We'll calculate the up vector always pointing as close towards the sky (which is to say, towards (0, 1, 0)) as possible. First make sure your forward vector is normalized. Then do this:

UP = (-Fx * Fy, Fx² + Fz², -Fy * Fz)
RIGHT = cross(UP, FORWARD)

with (Fx, Fy, Fz) being the forward vector. Normalize your up and right vector, plug all three of them in their respective columns in your rotation matrix, and you're done.

There's two drawbacks to this approach:

• It bugs out when your camera points perfectly up or down, so make sure that never happens by limiting your viewing angle.
• You can't easily 'roll' your camera around the z-axis (imagine tilting your head left or right).

Otherwise this should work fine.

Answer 2

As python code (assumes x to the right and y down in camera space):

def camera_rotation_matrix(pointing_direction: np.ndarray, up_vector: np.ndarray):
    forward = pointing_direction / np.linalg.norm(pointing_direction)
    right = np.cross(forward, up_vector)
    right /= np.linalg.norm(right)
    up = np.cross(forward, right)
    up /= np.linalg.norm(up)
    return np.column_stack((right, up, forward))