# problem with frustum AABB culling in DirectX

I am currently working on a project with a few friends, and I am trying to get frustum culling working. Every single tutorial or article I go to shows that my math is correct and that this should be working. I thought maybe posting here, somebody would catch something I could not. Thank you.

Here are the important code snippets

     /create the projection matrix
void CD3DCamera::SetLens(float fov, float aspect, float nearZ, float farZ)
{
D3DXMatrixPerspectiveFovLH(&projMat, D3DXToRadian(fov), aspect, nearZ, farZ);
}

//build the view matrix after changes have been made to camera
void CD3DCamera::BuildView()
{
//keep axes orthoganal
D3DXVec3Normalize(&look, &look);

//up
D3DXVec3Cross(&up, &look, &right);
D3DXVec3Normalize(&up, &up);

//right
D3DXVec3Cross(&right, &up, &look);
D3DXVec3Normalize(&right, &right);

//fill view matrix
float x = -D3DXVec3Dot(&position, &right);
float y = -D3DXVec3Dot(&position, &up);
float z = -D3DXVec3Dot(&position, &look);

viewMat(0,0) = right.x;
viewMat(1,0) = right.y;
viewMat(2,0) = right.z;
viewMat(3,0) = x;

viewMat(0,1) = up.x;
viewMat(1,1) = up.y;
viewMat(2,1) = up.z;
viewMat(3,1) = y;

viewMat(0,2) = look.x;
viewMat(1,2) = look.y;
viewMat(2,2) = look.z;
viewMat(3,2) = z;

viewMat(0,3) = 0.0f;
viewMat(1,3) = 0.0f;
viewMat(2,3) = 0.0f;
viewMat(3,3) = 1.0f;
}

void CD3DCamera::BuildFrustum()
{

D3DXMATRIX VP;
D3DXMatrixMultiply(&VP, &viewMat, &projMat);

D3DXVECTOR4 col0(VP(0,0), VP(1,0), VP(2,0), VP(3,0));

D3DXVECTOR4 col1(VP(0,1), VP(1,1), VP(2,1), VP(3,1));

D3DXVECTOR4 col2(VP(0,2), VP(1,2), VP(2,2), VP(3,2));

D3DXVECTOR4 col3(VP(0,3), VP(1,3), VP(2,3), VP(3,3));

// Planes face inward
frustum[0] = (D3DXPLANE)(col2);    // near
frustum[1] = (D3DXPLANE)(col3 - col2);  // far
frustum[2] = (D3DXPLANE)(col3 + col0);  // left
frustum[3] = (D3DXPLANE)(col3 - col0);  // right
frustum[4] = (D3DXPLANE)(col3 - col1);  // top
frustum[5] = (D3DXPLANE)(col3 + col1);  // bottom

// Normalize the frustum
for( int i = 0; i < 6; ++i )
D3DXPlaneNormalize( &frustum[i], &frustum[i] );

}

bool FrustumCheck(D3DXVECTOR3 max, D3DXVECTOR3 min, const D3DXPLANE* frustum)
{
// Test assumes frustum planes face inward.

D3DXVECTOR3 P;
D3DXVECTOR3 Q;
bool ret = false;

for(int i = 0; i < 6; ++i)
{
// For each coordinate axis x, y, z...
for(int j = 0; j < 3; ++j)
{
// Make PQ point in the same direction as the plane normal on this axis.
if( frustum[i][j] > 0.0f )
{
P[j] = min[j];
Q[j] = max[j];
}
else
{
P[j] = max[j];
Q[j] = min[j];
}
}

if(D3DXPlaneDotCoord(&frustum[i], &Q) < 0.0f  )
ret = false;
}
return true;
}

• you set the variable ret = true/false, but you just return true explicitly, is this a "typo" or actual code error? – Daemin Jan 18 '11 at 2:52
• sorry typo, its really a return false there instead of setting it to false – user4712 Jan 18 '11 at 6:33
• Thank you for looking at the code, everyone that did, I have fixed the problem. The problem was that our level editor's coordinate system was flipped on the x axis. So what we had to do was flip the x on the vectors of the min/max of the bounding boxes so that they were accurate. However, eventually we will go back and fix the level editor so that it has the correct coordinate system. lol – user4712 Jan 20 '11 at 7:21
• Could help if you put that in an answer and accept it in case anyone else runs into a similar problem :) – Ray Dey Jan 21 '11 at 18:25
• @Matthew Poole: what happens to P? You compute it but don't test it? – Will Jan 23 '11 at 19:04

## 1 Answer

Thank you for looking at the code, everyone that did, I have fixed the problem. The problem was that our level editor's coordinate system was flipped on the x axis. So what we had to do was flip the x on the vectors of the min/max of the bounding boxes so that they were accurate. However, eventually we will go back and fix the level editor so that it has the correct coordinate system. lol