I have been grappling with the Hartmann/Gribbs method of extracting the Frustum planes for some time now, with little success. There doesn't appear to be a "definitive" topic or tutorial which combines all the necessary information, so perhaps this can be it
First of all, I am attempting to do this in C# (For Playstation Mobile), using OpenGL style Column-Major matrices in a Right-Handed coordinate system but obviously the math will work in any language.
My projection matrix has a Near plane at 1.0, Far plane at 1000, FOV of 45.0 and Aspect of 1.7647.
I want to get my planes in World-Space, so I build my frustum from the View-Projection Matrix (that's projectionMatrix * viewMatrix). The view Matrix is the inverse of the camera's World-Transform.
The problem is; regardless of what I tweak, I can't seem to get a correct frustum. I think that I may be missing something obvious.
Focusing on the Near and Far planes for the moment (since they have the most obvious normals when correct), when my camera is positioned looking down the negative z-axis, I get two planes facing in the same direction, rather than opposite directions. If i strafe my camera left and right (while still looking along the z axis) the x value of the normal vector changes.
Obviously, something is fundamentally wrong here; I just can't figure out what - maybe someone here can?
EDIT: Applying the algorithm to just the Projection matrix (i.e. model and view matrices are identity) I get the Near and Far planes returned as the following; Far: normal[0,0-1], distance 1 Near: normal[0,0,-1], distance -0.3328891
Can someone tell me if these seem correct? Using the algorithm with just the Projection Matrix should return the planes in View Space (camera space)