I must say that I am really confused by how a view matrix is constructed and works.

First, there are 3 terms: view matrix, lookat matrix, and camera transformation matrix. Are those 3 the same, or different things. From what I undestand, the camera transformation matrix is basically the model matrix of the camera, and the view matrix is the inverse of that. The lookat matrix is basically for going from world space to view space, and I think I undestand how it works (doing dot products for projecting a point into another coordinate system).

I am also confused by the fact that sometimes, it seems like the view matrix is built by translation and dot products, and some other times, it is built with translation and rotation (with cos and sin).

There are also quaternions. When you convert a quaternion to a matrix, what kind of matrix is this?

Can someone explain to me how it really works, or point me towards a good resource.

Thank you.


1 Answer 1


Spaces and Matrices

First of all, as you know, we use matrices to represent and to do coordinate transformations. They can be any of the affine transformations. That includes translation, scaling, rotation, shear, and reflection transformations.

We (traditionally) have these coordinate systems to work with:

  • Model space (sometimes called "Object space"): The coordinates inside the model.
  • World space: The coordinates in the world.
  • Camera space: The coordinates relative to the camera.
  • Screen space (sometimes called "Window space" or "Device space"): The coordinates for the screen.

And of course, there are matrices to transform between them:

  • Model matrix (sometimes called “Object matrix”): from Model space to World space. You use this matrix to place objects in the world.
  • View matrix (sometimes called “Camera Transformation matrix”): from World space to Camera space.
  • Projection Matrix (Sometimes called “Camera Projection matrix”): from Camera space to Clip space.

Wait, Clip space? Will come back to that.

Of course, as you would know, we can compose matrices by multiplying them. We can also invert them, resulting in a matrix usable to do the transformation in the opposite direction.

To apply a transformation to a vector, you do a matrix-vector multiplication. Which is a matrix multiplication where one of the matrices happens to be a vector. And yes, the code looks like a bunch of dot products, it is all multiply and add, which is very fast for the computer to do. Also, yes, the values in rotation matrices come from trigonometric operations.

Note: Remember that matrix multiplication is not commutative. The order of the operands matters.

In fact, we use square (4D) matrices instead for augmented 3D matrices. This ensures that inverse always exists. So we augment the 3D vectors with a w component (usually with value 1), and then apply the transformation. This leaves a result that has a w coordinate which controls translation. Our 3D translation is an affine transformation in 3D, yet it is a linear transformation in 4D.

We usually do not want to have w or use it for scaled translations. So we normalize by multiplying the vector by 1/w, then discard w, leaving us a 3D vector. This is effectively a projection from 4D to 3D.

However, if we do not want the transformation to do any translation, we can augment the vector with a w component with value 0 instead.

See also Homogeneous Coordinates and Transformation Matrices.

So, as you can imagine, the result of transforming to Clip space is 4D. We do the above normalization by w, and that gives us a 3D space. This is called Normalized Device space (sometimes called “Normalized Screen space”).

Finally, the conversion from Normalized Device space to Screen space is controlled by the viewport (and depth range). Finally leaving us with a 2D space in pixels.

Virtual Camera

We also want to model a virtual camera. The simplest perspective virtual cameras have a position in the world, an orientation, and a field of view (will get to this one later).

  • Position:

    We are going to represent the position of the camera by using a vector in World space.

  • Rotation:

    Using director (Euler) angles is going to be easier to control for first person games. In particular for those games where there is a clear up and down. Which are most of them.

    If we need to compose rotations, interpolate rotations, or we want controls for open space (with no clear up and down), we want quaternions. And yes, you can have the input be director angles and then create a quaternion from those angles. Note: Similar to matrices, you also compose quaternions by multiplication. And similar to matrices, the order matters.

    If you want a camera that looks at something (for example a third person camera), you can use a Look-at matrix. You may want to convert it to quaternions for interpolation if needed to animate the camera rotating to look at something.

    You build your Look-at matrix with a forward vector which points in the direction towards where the camera is looking at, and an up vector. If you only had the forward vector, the camera rotation still has one degree of freedom (rotating around the forward vector)... The up vector fixes that. Thus, remember that the up vector and forward vector should not be in the same direction. Ideally they are perpendicular.

You pick what you use to represent the position and rotation of the camera for convenience. Regardless of what you use. You are going to convert them into matrices, so we can compose them for our render pipeline.

So, we will have a Camera Position matrix and a Camera Rotation matrix. If you are using a Look-at matrix for the rotation of the camera, that is the Camera Rotation matrix, no need to convert anything.

We are going to compose the Camera Position matrix and a Camera Rotation matrix into our Camera Transformation matrix (View matrix).

We still need to convert from Camera space to Screen Clip space. That is, we still need a Camera Projection matrix. The camera projection matrix is the one that would represent the camera frustum… that is, it is the one that would implement perspective. And it is the one that we create with the field of view of the camera.

Of course, the camera does not have to be a perspective camera. This is just the common case.

The Vertex shader is the opportunity we have to apply our Model matrix, View matrix and Projection matrix. Which you could pre-compose in a single Model-View-Projection matrix.

If you write your Vertex shader, you could decide to not have a Model matrix, View matrix and Projection matrix. You can do things differently. For example, for Ray tracing.

Your questions

First, there are 3 terms: view matrix, lookat matrix, and camera transformation matrix.

The View matrix converts from World space to Camera space. The Look-at matrix is usually used for the Camera Rotation matrix. And the Camera Transformation matrix is the Camera Position matrix composed with the Camera Rotation matrix.

I am also confused by the fact that sometimes, it seems like the view matrix is built by translation and dot products, and some other times, it is built with translation and rotation (with cos and sin).

Sometimes the Camera Rotation matrix (Look-at matrix if you are using that) is the identity matrix, resulting in a View matrix with no rotation.

There are also quaternions. When you convert a quaternion to a matrix, what kind of matrix is this?

At the end you are going to convert to matrices whatever representation of rotation you have, so you can apply the rotation transformation to vectors. However, you may want to do some operations between quaternions. In particular composition of rotations and interpolation of rotations.

What kind of matrix is it? It is a rotation matrix. If it is any of the (transitionally) named matrices depends on how you use it.

Bonus chatter

If you want to do skeleton animation, you would use transformation to moves parts in respect to others. That is also done with matrices, quaternions, etc. Perhaps a Look-at matrix can be useful to make a character point its arm in a particular direction. You may also be interested in inverse kinematics.

Learning material

You will find a good introduction to modern OpenGL for beginers at OGLdev.

I learned the old fashion way, with the red book. It should serve you well as reference and to clear up concepts.

I also want to recommend Learn OpenGL and opengl-tutorial.

For learning to code shaders, aside from the above, I found Shadertoy and Shaderific useful.

By the way, you are going to want a library that handles setting up the viewport (window or full-screen) and handling input... many (old) tutorials use glut. If you see glut (and you are going to) go use freeglut, it is a drop-in replacement... why?

The original GLUT library seems to have been abandoned with the most recent version (3.7) dating back to August 1998. Its license does not allow anyone to distribute modified library code. This would be OK, if not for the fact that GLUT is getting old and really needs improvement. Also, GLUT's license is incompatible with some software distributions (e.g., XFree86).

-- source.

Other alternatives include glfw or sdl. See GLUT-like Windowing, GUI, and Media Control toolkits. I'm currently in the glfw camp.

Note: OpenGL code is relatively easy to translate from a language to another. In particular if you are using a portable library to handle the setup of the viewport. Do not be afraid of reading a tutorial for a different programming language than the one you are using. In fact, I would argue that learning WebGL is a good idea. Despite it being a different API, it is very close. I recommend the MDN WebGl tutorial.

  • 3
    \$\begingroup\$ Hey, I'd like to congratulate you for your 10k rep on this site, and thank you (a lot) for your great contributions in helping users here :) \$\endgroup\$
    – Vaillancourt
    Commented Jan 28, 2020 at 4:27
  • 1
    \$\begingroup\$ "The View matrix converts from World space to Clip space" The view matrix transforms coordinates from world-space to camera/view space, not clip-space. To reach clip space, you apply a projection matrix to vertices in camera/view space. You need to correct that. \$\endgroup\$
    – code_dredd
    Commented Jul 18, 2021 at 22:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .