Finding translation component of a 4x4 matrix inverse

Question

I have 4x4 matrix (A) (maya softw):

|Xx Xy Xz 0| - basis of X and dummy 0
|Yx Yy Yz 0| - basis of Y and dummy 0
|Zx Zy Zz 0| - basis of Z and dummy 0
|Tx Ty Tz 1| - Position x, y, z and dummy 1

In this case, I know there is no scaling or sheer, so the upper-left 3x3 portion is a pure rotation matrix. That means I can invert this portion by taking the transpose.

So inverting the matrix should give me something like this:

|Xx Yx Zx 0|
|Xy Yy Zy 0|
|Xz Yz Zz 0|
|T? T? T? 1|

But I don't know how to find the T? values that correctly invert the translation component of the matrix. It's not just the inverted position (T-1)

---

My thinking so far:

Matrix of an untransformed object at the origin - (I - identity):

| 1  0  0  0 |
| 0  1  0  0 |
| 0  0  1  0 |
| 0  0  0  1 |

Matrix of a translated object (Tx=3, Ty=4, Tz=5) (A):

| 1  0  0  0 |
| 0  1  0  0 |
| 0  0  1  0 |
| 3  4  5  1 |

Inverse matrix of A (A'):

| 1  0  0  0 |
| 0  1  0  0 |
| 0  0  1  0 |
|-3 -4 -5  1 |

Matrix of translated and rotated(Y=90) object(B):

| 0  0 -1  0 |
| 0  1  0  0 |
| 1  0  0  0 |
| 3  4  5  1 |

inverse matrix of B (B'):

| 0  0  1  0 |
| 0  1  0  0 |
|-1  0  0  0 |
| 5 -4 -3  1 |

Strange things start to happen when you take two differently-rotated matrices with the same translation and you get different inverse translations!

It seems like some dot product is involved in the translation calculation.

Answer 1

If I have a matrix \$M\$ with translation vector \$\vec T\$, that means it maps the origin to the vector \$\vec T\$ in homogeneous coordinates, ie:

$$\begin{bmatrix}0 & 0 & 0 & 1\end{bmatrix} \cdot M = \begin{bmatrix}T_x & T_y & T_z & 1 \end{bmatrix} = \begin{bmatrix} \vec T & 1 \end{bmatrix}$$

So then the inverse of \$M\$, that is \$M^{-1}\$, should do the opposite, and map \$ \vec T\$ back to the origin.

We can think of the result of the matrix multiplication as a sum of the input vector multiplied by the first three rows (what the rotation / scaling / shearing part of the matrix do to the vector) plus the input vector multiplied by the final row (what the translation part of the matrix does to the vector). To get back to zero, this last part has to exactly negate what happened before.

So, if we look at just the upper 3x3 portion of your given inverse does to the vector \$\vec T\$...

$$\begin{bmatrix}T_x & T_y & T_z\end{bmatrix} \cdot \begin{bmatrix}X_x & Y_x & Z_x \\ X_y & Y_y & Z_y\\X_z & Y_z & Z_z \end{bmatrix} = \begin{bmatrix} \vec T \cdot \vec X & \vec T \cdot \vec Y & \vec T \cdot \vec Z\end{bmatrix}$$

This gives us what our inverse translation needs to undo. So our final translation vector, call it \$\vec U\$ for "Undo", is:

$$\vec U = -1 \cdot\begin{bmatrix} \vec T \cdot \vec X & \vec T \cdot \vec Y & \vec T \cdot \vec Z\end{bmatrix}$$

And your inverted 4x4 matrix is:

$$\begin{bmatrix}X_x & Y_x & Z_x & 0\\ X_y & Y_y & Z_y & 0 \\X_z & Y_z & Z_z & 0 \\ -\vec T \cdot \vec X & -\vec T \cdot \vec Y & -\vec T \cdot \vec Z & 1\end{bmatrix}$$