# How to make backface culling work correctly in both orthographic and perspective projection?

I have a software renderer that I've been building. I just implemented backface culling with the Go code below.

This works with Perspective Projection. But I plan to use orthographic projection. When switching to orthographic projection, the triangles that should be culled are culled longer than they should be.

I'm trying to figure out if I have a bug in my code or if I'm just not taking into account the projection.

My pipeline is basically:

1. Multiply each vertex by the world matrix
2. Backface cull triangles with the code below
3. Multiply each remaining vertex by the projection matrix

I've seen something about projecting before culling. I tried that though and it didn't help.

Is there something I'm not calculating to compensate for orthographic projection?

Perspective Projection (correct)

Orthographic Projection (incorrect)

// Culling logic
a, b, c := vertices[0], vertices[1], vertices[2]
ab := b.Sub(a)
ac := c.Sub(a)

normal := ab.Cross(ac).Normalize()
cameraRay := cameraPosition.Sub(a)

visibility := normal.Dot(cameraRay)
if visibility < 0.0 {
// don't render this triangle
}

• Hint: An orthographic camera isn't a point, but a whole plane. You want to cull based on whether the normals face towards/away from the plane, not a point on it. Commented Dec 15, 2022 at 16:53

Backface culling is not done based on normals. You can backface cull even meshes that have no normal information.

Backface culling is done using the winding order of the vertices' post-projection positions. That is, as you trace the shape of the triangle on the screen in the order the vertices were given, do you end up travelling around its perimeter clockwise or counter-clockwise?

sign = ab.x*ac.y - ac.x*ab.y;


The sign will be positive for faces wound in one direction, and negative for faces wound in the other direction.

This is similar to the cross product you're using to compute the normal, but notice that it uses only information in the image plane, no depth (z), and is insensitive to the position of the camera.

• Thanks! Every software renderer I’ve seen used the face normals for culling. Your method is obviously way less expensive to compute too. I’ve just never seen it before. I’ll try it out when I get back home. Commented Dec 15, 2022 at 2:33
• @AdamP, Using face normals for culling is typical for 3D model editors, where the winding order is adjusted automatically. Then, the face culling is done using the winding order. Using the winding order for culling is the “standard” way to do it. Commented Dec 15, 2022 at 9:38

The reason that your culling only works with perspective projection is that you are checking the angle (via dot product) between the ray from the camera position to the object and the triangle's normal. This works perfectly with perspective projection, but it's the wrong ray to compare for orthographic.

In general, the ray that you should compare against the triangle's normal (if you want to do it this way and not via winding order) is the ray which occupies a single point on screen — the ray which is oriented in the direction which the projection, whatever it is, flattens out of existence.

For perspective projection, all of those rays intersect at a point, typically considered the camera position, and you're correctly computing the relevant ray.

But for an orthographic projection, all of those rays are parallel and do not intersect.

(In a sense, an orthographic camera does not have a unique point which can be counted as its position; rather, it has a rectangle corresponding to the viewport. The point you consider its position only matters for what happens if you rotate the camera; without a difference in rotation, the relative position of a projected point to another projected point is always the same. Choosing a different center point just translates/crops the whole image.)

If you want to implement culling in the way you currently have working for perspective, but specifically for an orthographic projection, then compute the vector which is the camera's forward direction (and is the same regardless of where the triangle is on the screen), and use that in the dot product with the triangle normal. The camera's position will not be used — only its rotation.

• Hey thank you so much for this explanation. It helped a ton. I chose DMGregory's answer because that code is what actually helped me resolve it, but your answer made me actually understand the issue. Thanks for taking the time to help. Commented Dec 15, 2022 at 4:46
• The way I think about it, in orthographic projection the "camera" is a plane rather than a point.
– Ben
Commented Dec 16, 2022 at 1:03
• Actually, in orthographic projection the position of the camera does have a meaning: it defines the center point of the image. Shift your camera sideways, and the image also shifts in the opposite direction. Commented Dec 16, 2022 at 7:09
• @Ruslan Yes, but "center" is not a special point in orthographic projection (you could equally well define it starting from a corner if you wanted), and moving the camera is completely identical to changing the viewport. I've edited to clarify what I meant there. The important part is that it doesn't change the relationship of any two projected points to each other. Commented Dec 16, 2022 at 15:53
• I upvoted this answer because it's the one that gets to the root of the misunderstanding and solves it directly, even if there is a "better way" to do this. (Aside: I'm not sure the "better way" is actually better since it requires mapping into screen space, which requires a division per vertex, whereas the approach in this answer can be done on the untransformed geometry and thereby drop computations early). Commented Dec 16, 2022 at 19:01