Can someone explain the (reasons for the) implications of colum vs row major in multiplication/concatenation?

Question

I am trying to learn how to construct view and projection matrices, and keep reaching difficulties in my implementation owing to my confusion about the two standards for matrices.
I know how to multiply a matrix, and I can see that transposing before multiplication would completely change the result, hence the need to multiply in a different order.

What I don't understand though is whats meant by only 'notational convention' - from the articles here and here the authors appear to assert that it makes no difference to how the matrix is stored, or transferred to the GPU, but on the second page that matrix is clearly not equivalent to how it would be laid out in memory for row-major; and if I look at a populated matrix in my program I see the translation components occupying the 4th, 8th and 12th elements.

Given that:

"post-multiplying with column-major matrices produces the same result as pre-multiplying with row-major matrices. "

Why in the following snippet of code:

        Matrix4 r = t3 * t2 * t1;
        Matrix4 r2 = t1.Transpose() * t2.Transpose() * t3.Transpose();

Does r != r2 and why does pos3 != pos for:

        Vector4 pos = wvpM * new Vector4(0f, 15f, 15f, 1);
        Vector4 pos3 = wvpM.Transpose() * new Vector4(0f, 15f, 15f, 1);

Does the multiplication process change depending on whether the matrices are row or column major, or is it just the order (for an equivalent effect?)

One thing that isn't helping this become any clearer, is that when provided to DirectX, my column major WVP matrix is used successfully to transform vertices with the HLSL call: mul(vector,matrix) which should result in the vector being treated as row-major, so how can the column major matrix provided by my math library work?

Answer 1

if I look at a populated matrix in my program I see the translation components occupying the 4th, 8th and 12th elements.

Before I begin, it's important to understand: this means your matrices are row major. Therefore, you answer to this question:

my column major WVP matrix is used successfully to transform vertices with the HLSL call: mul(vector,matrix) which should result in the vector being treated as row-major, so how can the column major matrix provided by my math library work?

is quite simple: your matrices are row-major.

So many people use row-major or transposed matrices, that they forget that matrices are not naturally oriented that way. So they see a translation matrix as this:

1 0 0 0
0 1 0 0
0 0 1 0
x y z 1

This is a transposed translation matrix. That is not what a normal translation matrix looks like. The translation goes in the 4th column, not the fourth row. Sometimes, you even see this in textbooks, which is utter garbage.

It's easy to know whether a matrix in an array is row or column-major. If it's row-major, then the translation is stored in the 3, 7, and 11th indices. If it's column-major, then the translation is stored in the 12, 13, and 14th indices. Zero-base indices of course.

Your confusion stems from believing that you're using column-major matrices when you're in fact using row-major ones.

The statement that row vs. column major is a notational convention only is entirely true. The mechanics of matrix multiplication and matrix/vector multiplication are the same regardless of the convention.

What changes is the meaning of the results.

A 4x4 matrix after all is just a 4x4 grid of numbers. It doesn't have to refer to a change of coordinate system. However, once you assign meaning to a particular matrix, you now need to know what is stored in it and how to use it.

Take the translation matrix I showed you above. That's a valid matrix. You could store that matrix in a float[16] in one of two ways:

float row_major_t[16] =    {1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, x, y, z, 1};
float column_major_t[16] = {1, 0, 0, x, 0, 1, 0, y, 0, 0, 1, z, 0, 0, 0, 1};

However, I said that this translation matrix is wrong, because the translation is in the wrong place. I specifically said that it is transposed relative to the standard convention for how to build translation matrices, which ought to look like this:

1 0 0 x
0 1 0 y
0 0 1 z
0 0 0 1

Let's look at how these are stored:

float row_major[16] =    {1, 0, 0, x, 0, 1, 0, y, 0, 0, 1, z, 0, 0, 0, 1};
float column_major[16] = {1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, x, y, z, 1};

Notice that column_major is exactly the same as row_major_t. So, if we take a proper translation matrix, and store it as column-major, it is the same as transposing that matrix and storing it as row-major.

That is what is meant by being only a notational convention. There are really two sets of conventions: memory storage and transposition. Memory storage is column vs row major, while transposition is normal vs. transposed.

If you have a matrix that was generated in row-major order, you can get the same effect by transposing the column-major equivalent of that matrix. And vice-versa.

Matrix multiplication can only be done one way: given two matrices, in a specific order, you multiply certain values together and store the results. Now, A*B != B*A, but the actual source code for A*B is the same as the code for B*A. They both run the same code to compute the output.

The matrix multiplication code does not care whether the matrices happen to be stored in column-major or row-major order.

The same cannot be said for vector/matrix multiplication. And here's why.

Vector/matrix multiplication is a falsehood; it cannot be done. However, you can multiply a matrix by another matrix. So if you pretend a vector is a matrix, then you can effectively do vector/matrix multiplication, simply by doing matrix/matrix multiplication.

A 4D vector can be considered a column-vector or a row-vector. That is, a 4D vector can be thought of as a 4x1 matrix (remember: in matrix notation, the row count comes first) or a 1x4 matrix.

But here's the thing: Given two matrices A and B, A*B is only defined if the number of columns of A is the same as the number of rows of B. Therefore, if A is our 4x4 matrix, B must be a matrix with 4 rows in it. Therefore, you cannot perform A*x, where x is a row-vector. Similarly, you cannot perform x*A where x is a column-vector.

Because of this, most matrix math libraries make this assumption: if you multiply a vector times a matrix, you really mean to do the multiplication that actually works, not the one that makes no sense.

Let us define, for any 4D vector x, the following. C shall be the column-vector matrix form of x, and R shall be the row-vector matrix form of x. Given this, for any 4x4 matrix A, A*C represents matrix multiplying A by the column-vector x. And R*A represents matrix multiplying the row-vector x by A.

But if we look at this using strict matrix math, we see that these are not equivalent. R*A cannot be the same as A*C. This is because a row-vector is not the same thing as a column-vector. They're not the same matrix, so they do not produce the same results.

However, they are related in one way. It is true that R != C. However, it is also true that R = CT, where ^T is the transpose operation. The two matrices are transposes of each other.

Here's a funny fact. Since vectors are treated as matrices, they too have a column vs. row-major storage question. The problem is that they both look the same. The array of floats is the same, so you can't tell the difference between R and C just by looking at the data. The only way to tell the difference is by how they are used.

If you have any two matrices A and B, and A is stored as row-major and B as column-major, multiplying them is completely meaningless. You get nonsense as a result. Well, not really. Mathematically, what you get is the equivalent of doing AT*B. Or A*BT; they're mathematically identical.

Therefore, matrix multiplication only makes sense if the two matrices (and remember: vector/matrix multiplication is just matrix multiplication) are stored in the same major ordering.

So, is a vector column-major or row-major? It is both and neither, as stated before. It is column major only when it is used as a column matrix, and it is row major when it is used as a row matrix.

Therefore, if you have a matrix A which is column major, x*A means... nothing. Well, again, it means x*AT, but that's not what you really wanted. Similarly, A*x does transposed multiplication if A is row-major.

Therefore, the order of vector/matrix multiplication does change, depending on your major ordering of the data (and whether you're using transposed matrices).

Why in the following snippet of code does r != r2

Because your code is broken and buggy. Mathematically, A * (B * C) == (CT * BT) * AT. If you do not get this result, then either your equality test is wrong (floating-point precision issues) or your matrix multiplication code is broken.

why does pos3 != pos for

Because that doesn't make sense. The only way for A * t == AT * t to be true would be if A == AT. And that is only true of symmetric matrices.

Answer 2

There's two different choices of convention at work here. One is whether you use row vectors or column vectors, and the matrices for these conventions are transposes of each other.

The other is whether you store the matrices in memory in row-major order or column-major order. Note that "row-major" and "column-major" are not the correct terms for discussing the row-vector / column-vector convention...even though many people misuse them as such. The row-major and column-major memory layouts differ by a transpose, too.

OpenGL uses a column vector convention and column-major storage order, and D3D uses a row vector convention and row-major storage order (well - at least D3DX, the math library, does), so the two transposes cancel out and it turns out the same memory layout works for both OpenGL and D3D. That is, the same list of 16 floats stored sequentially in memory will work the same way in both APIs.

This may be what's meant by people saying that "it makes no difference to how the matrix is stored, or transferred to the GPU".

As for your code snippets, r != r2 because the rule for transpose of a product is (ABC)^T = C^T B^T A^T. Transposition distributes over multiplication with a reveral of order. So in your case you should get r.Transpose() == r2, not r == r2.

Likewise, pos != pos3 because you transposed but didn't reverse the multiplication order. You should get wpvM * localPos == localPos * wvpM.Tranpose(). The vector is automatically being interpreted as a row vector when multiplied on the left side of a matrix, and as a column vector when multiplied on the right side of a matrix. Other than that, there is no change in how multiplication is carried out.

Finally, re: "my column major WVP matrix is used successfully to transform vertices with the HLSL call: mul(vector,matrix)," I'm uncertain about this, but maybe confusion / a bug has caused the matrix to come out from the math library already transposed.

Answer 3

In 3d graphics you use matrix to transform both vector and points. Considering the fact that you are talking about translation matrix, I'll speak only about points (you can't translate a vector with a matrix, or , saying it better, you can but you will obtain the same vector).

In matrix multiplication the number of column of the first matrix should be equal to the number of row of the second one (you can multiply a n x m matrix for an m x k).

A point (or a vector) is represented by 3 components (x,y,z) and can be considered both like a row or a column:

colum (dimension 3 X 1):

|x|

|y|

|z|

or

row (dimension 1 X 3):

|x,y,z|

You can choose the preferred convention, it's just a convention. Let's call it T the translation matrix. If you choose the first convention, in order to multiply a point p for a matrix you need to use a post multiplication:

T * v (dimension 3x3 * 3x1)

otherwise:

v * T (dimension 1x3 * 3x3)

the authors appear to assert that it makes no difference to how the matrix is stored, or transferred to the GPU

If you use always the same convention it makes no difference. It doesn't mean that matrix of different convention will have the same memory representation, but that transforming a point with the 2 different conventions you will obtain the same transformed point:

p2 = B * A * p1; //first convention

p3 = p1 * A * B; //second convention

p2 == p3;

Answer 4

I see the translation components occupying the 4th, 8th and 12th elements means your matrices are "wrong".

The translation components are always specified as entries #13, #14 and #15 of the transformation matrix (counting the very first element of the array as element #1).

A row major transformation matrix looks like this:

[ 2 2 2 1 ]  ┌ R00 R01 R02 0 ┐ 
             │ R10 R11 R12 0 │
             │ R20 R21 R22 0 │
             └ t.x t.y t.z 1 ┘

A column major transformation matrix looks like this:

┌ R00 R01 R02 t.x ┐ ┌ 2 ┐ 
│ R10 R11 R12 t.y │ │ 2 │
│ R20 R21 R22 t.z │ │ 2 │
└  0   0   0   1  ┘ └ 1 ┘

Row major matrices are specified going down the rows.

Declaring the row major matrix above as a linear array, I would write:

ROW_MAJOR = { R00, R01, R02, 0,  // row 1 // very intuitive
              R10, R11, R12, 0,  // row 2
              R20, R21, R22, 0,  // row 3
              t.x, t.y, t.z, 1 } ; // row 4

That seems very natural. Because notice, English is written "row-major" -- the matrix appears in the text above exactly as it will be in math.

And here's the point of confusion.

Column major matrices are specified going down the columns

That means to specify the column major transformation matrix as a linear array in code, you'd have to write:

    COLUMN_MAJOR = { R00, R10, R20, 0, // COLUMN #1 // very counter-intuitive
                     R01, R11, R21, 0,
                     R02, R12, R22, 0,
                     t.x, t.y, t.z, 1  } ;

Note this is completely counter-intuitive!! A column major matrix has its entries specified down the columns when initializing a linear array, so the first line

COLUMN_MAJOR = { R00, R10, R20, 0,

Specifies the first column of the matrix:

┌ R00
│ R10
│ R20
└  0 

and not the first row, as the simple layout of the text would have you believe. You have to mentally transpose a column major matrix when you see it in code, because the first 4 elements specified actually describe the first column. I suppose this is why a lot of people prefer row-major matrices in code (GO DIRECT3D!! cough.)

So, the translation components are always at linear array indices #13, #14, and #15 (where first element is #1), regardless if you're using row major or column major matrices.

What happened with your code and why does it work?

What's happening in your code is, you have a column major-matrix yes, but you put the translation components in the wrong spot. When you transpose the matrix, entry #4 goes to entry #13, entry #8 to #13, and entry #12 to #15. And there you have it.

Answer 5

Put simply, the reason for the difference is that matrix multiplication is not commutative. With regular multiplication of numbers, if A * B = C then it follows that B * A also = C. This is not the case with matrixes. That's why choosing either row-major or column-major matters.

Why it doesn't matter is that, in a modern API (and I'm specifically talking shaders here), you can choose your own convention and multiply your matrixes in the correct order for that convention in your own shader code. The API no longer enforces one or the other on you.