How can I retrieve the camera's world-space position from its view matrix? The only answers I've seen to this question suggest the translation is in the last row/col but this wouldn't work since the matrix contains [x (dot) right, y (dot) up, z (dot) look]
Other answers here explain how to get the camera position from camera matrix inverse.
If the 3x3 part of the camera matrix has only rotation (no scaling or shearing) as they usually do, the calculation can be optimized by multiplying the camera matrix translation with the transpose of the camera rotation. Camera position is then the transformed translation vector multiplied by -1. In GLSL this is:
This is what I have been using in my vertex shaders if I don't want to calculate and pass the camera position as a uniform.
First off I would HIGHLY recommend just storing the position as a vector separately, it'll make things much easier computationally. Anyways...
where the top left 3x3 matrix represents rotations, scale, etc. All the camera's orientation is done there. The remaining row and column are used for translation and some other complicated perspective stuff I'm not going to get into right now.
When you get the matrix (assuming it's a 4x4 matrix), translation will always be stored in either the last row or the last column, depending on whether your matrix class is row-major or column-major ordering.
What you're probably getting confused about is the fact that you need the dot products. What's going on is simplification of the matrix math, there are more detailed answers in this Stack Overflow question: http://stackoverflow.com/questions/349050/calculating-a-lookat-matrix
The solution can be found here, you need to take the inverse of the matrix and get the translation of that:
First invert the view matrix. Then fetch the translation from the last row/column.
One way to deduce the contents of a view matrix is to start by considering the camera as any other object in the world, and calculating a world matrix for it:
A world matrix transforms coordinates from local space to world space. But in this case, the camera's local space and the view space are one and the same, so we can also say this matrix transforms coordinates from view space to world space.
Since we need a conversion in the opposite direction, we have to invert the matrix. The result is what we call the view matrix which transforms coordinates from world space to view space:
And that's the sort of matrix you have. So in order to get the camera position back from it, you will first need to invert it, and then you can grab the translation from the last row (or column depending on the system).