What does a matrix represent?

Question

I began learning OpenGL recently and am having problems visualizing what matrices are and their role in computer graphics. Given the template of a 4x4 matrix like this:

I would assume that each matrix like this are the coordinates of a vertex in world space. And several of them put together and shaded give an object?

But why is there a Xx, a Xy and an Xz? I read that its a different axis (up, left, forward) but still can't make heads or tails of the significance.

Answer 1

Matrices in computer graphics are the transformations given to each coordinate in the model. Each Matrix is a combination of multiple transformations to apply to a coordinate (a point in 3-space).

Building a transformation is based from one of three transform types: Translate, Rotate and Scale.

A translation matrix is something like:

And a scale matrix:

Rotation matrices look like:

To combine any of these matrices you just multiply them together. To apply the transformation to a vertex just multiply to the vertex (as seen in the translation diagram).

Answer 2

In computer graphics, we use matrices to encode transformations.

Matrices that contain only translation, rotation or scaling transforms have a commonly-exploited interpretation: the upper-left 3x3 of the matrix contains only rotation or scale data, the bottom row or right column contains translation data. This is not a generality, but holds true often enough for the subset of transformations represented in computer graphics that people make use of it.

There is, similarly, a relationship between the values of the matrix and the corresponding coordinate frame the matrix represents (which is not always "world space," I should note). The upper-left 3x3 columns (or rows) represent the X, Y and Z axes of the coordinate frame.

Whether or not the rows represent axes or the column do depends on whether you are using the convention of multiplying as row vector * matrix or matrix * column vector. When performing matrix multiplication, the inner dimensions of the two matrices must agree, and so whether you are representing vectors as row matrices or column matrices impacts that choice (OpenGL and traditional math tend to prefer column vectors).

I recommend getting a good book on linear algebra, or at least taking a look at the Matrix and Quaternion FAQ and this post on matrix layouts in DirectX and OpenGL.

Answer 3

What is a matrix?

A matrix with m columns and n rows represents a function which consumes a vector^* with m elements (or coordinates) and produces a vector with n elements.

From this you can observe that if and only if a matrix is square, will the dimensionality of the vector not change. Eg. you get a 3D vector from transforming a 3D vector, a 2D from a 2D, etc.

^*: In physics, vectors are usually used to indicate forces or other "influences" which "move around" things such as velocity or acceleration. But there is nothing stopping you from using a vector to represent a point or any arbitrary array of numbers (some libraries and programming languages even use "vector" to mean "1D array"). For use with matrices, anything can be the elements of your vector (even strings or colors), so long as you have a way of adding, subtracting and multiplying them by whatever the elements of your matrix are. Hence the name vector, which means "carrier" - it carries or holds values for you.

What does multiplying by a matrix mean?

So if a matrix is a function, what kind of function? What does the function do? The recipe for it is defined by the elements of the matrix. Let's call the input u, output v, the matrix M (the multiplication M*u=v is then the same as f(u)=v) and u(i) gives the ith element of u (2nd element is y coordinate, for instance). For the matrix, M(i,j) means row i, column j.

Construction of element v(1), the first one in the result, is described by the first row of the matrix. u(1) times M(1,1), plus u(2) times M(1,2), ... plus u(i) times M(1,i). A matrix is a bit like a very simple programming language, that is only good for programming functions that work by shuffling around the inputs, adding them to themselves, etc.^**

It is helpful to imagine that you are working on one element of output at a time, hence, you are using only one row of the matrix at a time. You write out u horizontally. You write the ith row of M below it. You multiply every above/below pair and write the products below, then add up the products. Repeat for every row to get every element of v. (Now you see why an m by n matrix must operate on an m vector and produce an n vector.)

Another way to think about this - let's say we are doing a 3D to 3D transform, so a 3x3 matrix (or 3D transform as they are often called because you can pretend this "function" is "moving" 3D points, even though really it's just changing up the numbers). Let's say the first row is [1 2 0]. This means, to get x of result, get 1 of input's x, 2 of input's y, and 0 of input's z. So it's really a recipe.

^**: If a matrix is a programming language, then it is not even Turing complete.

What does multiplying two matrices mean?

If they are both matrices of appropriate size, then A*B means "a function which applies first B then A". You can see why the constraints on sizes for multiplication exist, because size determines input and output size, and one matrix consume the output of the other. Why does multiplication mean combine functions? It's easier to notice that it has to be. If A*u is the same as f(u) and B*u is the same as g(u) then f(g(u)) is the same as f(B*u) which is the same as A*(B*u).

Likewise, repeated applications of the same function can be shown as powers, since A*A*A means applying the function that A represents thrice.

How are matrices useful?

What good is doing a transformation like new_x = 1*x+2*y+0*z (if the first row is [1 2 0])? That's not very obvious, but let's take another 2D matrix to explain that. The matrix is:

[ 0 1
  1 0 ]

Or [0 1; 1 0] using the convenient Matlab notation. What does this matrix do? It transforms a 2D vector like so: For the x of result, take 1 of the y of the input. For the y of result, take 1 of the x of the input. We've just swapped the x and y coordinates of the input - this matrix reflects points about the x=y line. That's sort of useful! By extension, you'll see that all matrices with 1s along the SW-NE line reflect. You can also see why identity matrices give you the input back (for x of output, take x of input; for y of output, take y of input...).

Now you see why the symbols are eg. Xx, Yx - they mean how much of input X, Y, etc. goes into output x.

How else are matrices useful?

What other transformation can you do? You can resize by taking an identity matrix, but with a different number than 1 along the diagonal. For example, [2.5 0; 0 22.5] will multiply every coordinate of the input by 2.5, and if you apply this matrix to every point in a picture, the picture will be 2.5 as large. If you only put a 2.5 in one row ([2.5 0; 0 1]) then only the x coordinate will be multiplied, so you will only stretch along x.

Other matrices can give other transformations, such as "skewing", which have varying degrees of usefulness. Personally, skew is my least favorite because the matrix looks so simple but the transform itself rarely does anything except mangle a picture. A useful one is "rotation" - how do you rotate a point? Try working out the position of point (x, y) after rotating by theta degrees counterclockwise about the origin. You will find that the new x and y coordinates both come out from multiplying the old x and y by some sines and cosines of theta. You should be able to easily write a rotation matrix using sines and cosines which corresponds to this function.

With non-square matrices, you can also change the dimensionality of an input. Turning a 2D input into 3D isn't very useful, since it's hard to "manufacture" something to put into the new coordinate, but 3D into 2D is very useful. Among other things, this is how your computer knows to project^*** a 3D scene into 2D image to draw on your monitor.

Since vectors can hold different things, you could even describe a matrix which encrypts a string n-characters at a time, by shuffling them around or "multiplying" them (you would have to come up with multiplication/addition function).

^***: When you project, you take a 3D object like a sculpture, shine a light on it, and see what kind of 2D shadow drops on a wall.

What are the limitations of matrices?

Can you do every function with matrices? No. Thinking graphically, it's hard to imagine something that a matrix couldn't do (but it exists: a "swirl" effect cannot be done, for instance). However, here's an easy example: Let's say function f is such that f(u) gives you back u with every element squared. You will see that you cannot write a matrix for this: With matrices there is only facility for describing recipes that multiply coordinates by a constant number, no other fancy functions like power can be expressed.

^****: This is also why it's called linear algebra - the power function is non-linear, it doesn't make a straight line when plotted.

On the weird extra row in 4D matrices

Now, why is the matrix in your example 4 by 4? Doesn't this mean 4-dimensional space? We don't have 4D computers, so why? This is actually an interesting trick with matrices that relates to the previous point about linear operations.

Regarding which functions cannot be done with matrices: What is the matrix for moving a 2D point by 2 units to the right (which produces the point (x+2, y)? Again, we get stuck. There is a way to multiply the input, but no way to add a constant. For 2D work, the trick is to pretend you are actually not in 2D space, but in 3D space, except the height (z coordinate or 3rd element) of everything is always 1 (it's a bit like how a 2D universe is just a "plate" lying flat along the floor of a 3D universe - in that case the third coordinate is always 0). Then you can use this magic last coordinate as a constant, because you know it is always 1 for every input.

Likewise, for moving 3D points, you need 4D coordinates. That's also why all 3D transformation matrices you see will have [0 0 0 1] as the last row - you must never alter the 4th dimension, or the result will be too complicated to represent in 3D!

Answer 4

That is a 4x4 column-major matrix, and from the looks of it, a view matrix.

The first 3 columns define direction of your basis vectors (up, left, forward as you called them), and the last column defines translation of the eye point. Put them together and you can describe the orientation of your camera, and more importantly you can use this matrix to transform points into a coordinate space known as "eye space", "view space" or "camera space".

Those are all synonyms for the same coordinate space. Unfortunately you have to learn all of the synonyms when dealing with computer graphics because different books and people will call them by different names. Most coordinate spaces have multiple names.

By the way, the three columns in your view matrix are generally orthogonal, that is they form right angles to one another. This is not required, but it is a very common property when constructing a traditional camera.

Answer 5

TL;DR version:

The first three elements [x y z] in each row represent a single basis vector of a transformed cooordinate system. The last element w is a translation component.

The Long Version

If you wanted a matrix that, when applied to a vertex, would rotate the vertex about the origin by, say, 45 degrees, you would fill the matrix with three vectors representing the transformed axes:

• A point i on the x axis [1 0 0], but rotated 45 degrees. This is simply [i_x i_y i_z], where i_x and i_y are the legs of a triangle with a 45-degree interior angle relative to the X axis: [cos(45) sin(45) 0].
• A point j on the y axis [0 1 0], but rotated 45 degrees from that axis. Sketch it on a piece of paper and you'll see that, when rotating counter-clockwise, the components become [-sin(45) cos(45) 0].
• A point k on the z axis. In this example, z is unaffected since we're rotating in the (screen-aligned) x-y plane

So, we have three new vectors: i,j, k. The easy way to visualize this is just taking the X and Y axes and rotating the whole cross arrangement.

How do we put these in a matrix?

i_x i_y i_z
j_x j_y j_z
k_x k_y k_z

Or

 cos(45)  sin(45)    0
-sin(45)  cos(45)    0
    0        0       1

If you multiply any vertex by that matrix, you'll get

v1_x = v_x cos(Θ)     - v_y sin(Θ) + v_z * 0
V1_y = v_x*sin(Θ)    + v_y cos(Θ) + v_Z * 0
V1_z = v_x * 0        + v_y * 0    + v_z * 1

for v = [1 0 0], and Θ = 90°, this becomes v1 = [0 1 0]

For translation, we add a fourth row and column, and put the translation components in the last column. We add a fourth component to the vertex w which is usually 1. This is so that, when we multiply the vertex by the matrix, the w component causes the last column to be added to the input vertex, so the vertex is moved or translated. We call these "homogeneous coordinates." (For our purposes, "homogeneous" just means that there is a 4th component w in each vector, and we use a 4x4 matrix instead of a 3x3. Frequently you'll see shaders that use 4x3 matrices to avoid sending the mostly-useless 4th row to the GPU, which consumes valuable memory and bandwidth. The 4th row is needed for perspective projection, but not much else.)

Hope this helps.