Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

I'm trying to get basic frustum culling against an AABB working, and I'm having a bit of trouble figuring out how to extract the frustum planes from my camera's transform matrix. All the example code I've found expects position, look, and up vectors, as you'd pass to gluLookAt. My camera matrix can be transformed by a parent and other bits of code, so those vectors never exist anywhere that I could use them.

I've looked at code for gluLookAt and tried to graft the inverse of that on top of the Lighthouse3D example for creating the Frustum planes. It's really close to correct, but I have objects flickering in and out of existence still, so something's not quite right. All of the related code is here, so I'm hoping this will be an easy find for somebody.

struct Frustum {
  enum {
    TOP,
    BOTTOM,
    LEFT,
    RIGHT,
    NEAR,
    FAR
  };

  Frustum(hsMatrix44 cam, float znear, float zfar, float angle, float aspect) {
    float nw, nh, fw, fh, tang;
    tang = (float)tan(angle * 0.5);
    nh = znear * tang;
    nw = nh * aspect; 
    fh = zfar  * tang;
    fw = fh * angle;

    hsVector3 dir,nc,fc,X,Y,Z,P,U;
    hsVector3 ntl,ntr,nbl,nbr,ftl,ftr,fbl,fbr;

    Z.X = -cam(2,0);
    Z.Y = -cam(2,1);
    Z.Z = -cam(2,2);
    U.X = cam(1,0);
    U.Y = cam(1,1);
    U.Z = cam(1,2);
    X = U.crossP(Z);
    X = X * (1.f/X.magnitude());
    Y = Z.crossP(X);
    hsMatrix44 icam = cam.inverse();
    P.X = icam(0,3);
    P.Y = icam(1,3);
    P.Z = icam(2,3);
    nc = P - Z * znear;
    fc = P - Z * zfar;

    ntl = nc + Y * nh - X * nw;
    ntr = nc + Y * nh + X * nw;
    nbl = nc - Y * nh - X * nw;
    nbr = nc - Y * nh + X * nw;
    ftl = fc + Y * fh - X * fw;
    ftr = fc + Y * fh + X * fw;
    fbl = fc - Y * fh - X * fw;
    fbr = fc - Y * fh + X * fw;

    planes[TOP] = planeFromPoints(ntr, ntl, ftl);
    planes[BOTTOM] = planeFromPoints(nbl, nbr, fbr);
    planes[LEFT] = planeFromPoints(ntl, nbl, fbl);
    planes[RIGHT] = planeFromPoints(nbr, ntr, fbr);
    planes[NEAR] = planeFromPoints(ntl, ntr, nbr);
    planes[FAR] = planeFromPoints(ftr, ftl, fbl);
  }

  bool checkBox(const hsBounds3Ext& box) {
    for(int i = 0; i < 6; ++i) {
      float d0, d1;
      d0 = planes[i].W + planes[i].N.dotP(box.getMins());
      d1 = planes[i].W + planes[i].N.dotP(box.getMaxs());
      if((d0 < 0.f) && (d1 < 0.f))
        return false;
    }
    return true;
  }

  hsPlane3 planes[6];
};

hsPlane3 planeFromPoints(hsVector3 a, hsVector3 b, hsVector3 c) {
  hsPlane3 p;
  hsVector3 tmp0, tmp1;
  tmp0 = a - b;
  tmp1 = c - b;
  p.N = tmp1.crossP(tmp0);
  p.N = p.N * (1.f/p.N.magnitude());
  p.W = -(p.N.dotP(b));
  return p;
}
share|improve this question
    
You mention only view matrix. Do you consider perspective matrix in your code? Means ... how do you define angle in this function: Frustum(hsMatrix44 cam, float znear, float zfar, float angle, float aspect). Do you use correct Fov and aspect? –  Notabene Jan 24 '11 at 2:52
1  
There have been several frustum/aabb culling code questions recently; would some expert make a definitive how (not why) guide that shows the steps e.g. how/ when to make the frustum, what transform to use on the things to test etc? –  Will Jan 24 '11 at 6:00
    
@notabene: the fov and apsect passed to my Frustum constructor match the ones used to generate my perspective matrix. I didn't see any reason to complicate things further trying to extract them from a perspective matrix when they were relatively easy to forward –  Branan Jan 24 '11 at 14:24
    
@Will: I don't think there is such a thing as a definitive tutorial. There's nothing special about a frustum, it's just another application of linear algebra and matrices. The problem is that matrices are like burritos (byorgey.wordpress.com/2009/01/12/…). –  user744 Jan 24 '11 at 14:50
add comment

1 Answer

I am using a function to extract the frustum from my Projection and modelview Matrix. This code was developed by a friend of mine using some Maple 5 or something like that. The code is a bit bulky (~10K) and the comments are in german, but it works fine ( i use it for octree-frustum culling in my OpenGL-Application).

The project is hosted on Sourceforge ( http://sourceforge.net/projects/sumkindaproject/ ).

The frustum-extraction and checking code is located in the folders inc/frustum.h and src/frustum.cpp

Please don't mind the nasty coding style in other files of this project, since it is more or less an experimental playground app - the Culling however works fine ;)

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.