Set-up: 3D world, isometric sprite rendering (4 fixed angles). This transform matrix is used as camera:
_transform =
Matrix.CreateRotationZ(Roll) *
Matrix.CreateRotationX(Pitch) *
Matrix.CreateTranslation(new Vector3(-cameraPosition.X , -cameraPosition.Y, -cameraPosition.Z))* //this matrix controls camera position
Matrix.CreateScale(new Vector3(_zoomScale)) * //scale representing zoom level. This one applies to the textures too
Matrix.CreateScale(new Vector3(_textureScale)) * //world scale to fit current texture pixel size.
Matrix.CreateTranslation(new Vector3((_port.screenWidth / 2) - TileOffset, _port.screenHeight / 2, 0)) //offset for having focus in center of screen instead of top left corner + little offset so it will be a tile center
;
Roll=-45, Pitch=60. Result:
I've hoped to find a quick picking solution in the inverse matrix:
_inverseTransform =
Matrix.CreateTranslation(new Vector3(-(_port.screenWidth / 2) + TileOffset, -_port.screenHeight / 2, 0))*
Matrix.CreateScale(new Vector3(1/_textureScale)) *
Matrix.CreateScale(new Vector3(1/_zoomScale)) *
Matrix.CreateTranslation(new Vector3(cameraPosition.X, cameraPosition.Y, cameraPosition.Z))*
Matrix.CreateRotationX(-Pitch)*
Matrix.CreateRotationZ(-Roll)
;
It doesn't work as expected. If i remove rotation from matrix build, everything works. The interesting part is if i leave roll component only, everything works too. I've tried transposed matrices, inverted matrices, rotating on -angle, FromPitchYawRoll matrix (inverted/transposed/-angles). Every time roll works, pitch and yaw aren't. Pitch only/yaw only can't be inverted too.
It's probably something basic in the matrix math/rotation principles, but i'm failing to grasp it. Any help would be appreciated.
--code removed, not relevant to the case--
Some gifs for better understanding. Current implementation do this:
If we switch the pitch off, inverse matrix work as intended (white dots mark real tiles grid. Graphical tiles aren't rotated, of course.)
EDIT: After some matrix calculations i've realised simple thing. Let's point the mouse on the tile (0,2). If i dump all transformations but pitch by several degrees, then tile (0,2) will be drawed somewhere on (0, 1.25).
But if i'll apply inverted (or transposed, or "rotate by -value") matrix to the mouse position (0,2), i'll get same (0, 1.25), which means that the mouse is pointing at (0,1) tile. While it should point at (0, 3). And i don't know how to invert this correctly.
I've also discovered this. But, as you can see in the first answer, knight666 is using scale instead of pitch rotation.