# How do game engines avoid recalculating normals upon mesh rotation?

Assume that a face of a mesh is defined by vertices A, B, C and that the normal of the face is defined by (B - A) x (C - A).

Rotating the mesh about its local origin with matrix R makes the normal calculation R(B - A) x R(C - A).

To my knowledge there is no algebraic property of the cross product that lets us simply do R((B - A) x (C - A)) to rotate the normal just like all the vertices. The cross product preserves distribution over an orthonormal matrix.

Instead it seems like the normal needs to be entirely recalculated every time the object is rotated. This seems unreasonably expensive for a realtime application so I was curious if game engines have some way to efficiently transform normals, approximate them, or avoid them altogether.

They don't.

When you rotate a mesh, the normals in memory are not changed in any way. All that changes is the object's transformation matrix is updated. The raw mesh vertex and normal data remains unchanged, and can be shared between many instances of the model that get drawn with different positions, orientations, and scales.

These transformations are applied "just in time" during rendering. We need to multiply the vertex positions by the camera's view and projection matrices anyway to figure out where to draw it on screen, so we can just pre-multiply those with the object's transformation matrix (often called either the model matrix or the world matrix) to get a combined "MVP" matrix (model-view-projection), then transform each vertex with just a single matrix multiplication in the vertex shader, as it's being drawn.

The same trick works for normals, it's just that we use the inverse-transpose of the model matrix multiplied by the view matrix (no projection) to map the normal into view space (where we usually want it in deferred rendering, though some lighting systems use a different coordinate space for lighting).

This isn't unduly expensive. A matrix multiplication per vertex isn't breaking the bank when we can do hundreds of them in parallel on modern GPUs, and we'd need to pay that cost anyway every time the camera moves, which in many games is already happening every frame. The ability to combine multiple matrix transformations into a single matrix lets us bake the model's transformation in "just in time" at no added cost per vertex.

For things like CPU-side physics calculations, we can often use the inverse of the model matrix to transform the calculation into the model's local space, rather than transforming all vertex positions and normals into world space. Then once we know which face has been collided with, we can take the one resulting collision normal and transform just it into world space, without having to touch any of the others.

• Thanks for the detailed answer! So from what I gather about the inverse-transpose matrix, if the transformation matrix R is orthonormal, then we can use R((B - A) x (C - A)) to rotate the normal regardless of whether or not it is stored or computed on the fly? Oct 17, 2023 at 3:33
• We generally don't do those cross products on the fly. Normals are calculated or manually tweaked by the artist in their 3D content creation software, and saved as a channel of data alongside vertex positions, texture coordinates, etc. This is important because it allows artists to control which edges should be smooth vs hard, or implement custom shading effects with specially tweaked normals. Cross product normals would mainly be used for procedural geometry. But yes, if the matrix is orthonormal, its inverse and transpose are the same, so it is its own inverse transpose. Oct 17, 2023 at 3:46
• The lead sentence ("we don't") looks strange in response to the question title ("how do game engines avoid recalculating ..."). Oct 17, 2023 at 18:29
• We do not avoid recalculating normals. We re-calculate the normal's transformation by the model and view matrices every frame. So the "they don't" is very deliberate. Oct 17, 2023 at 19:32
• "This isn't unduly expensive" Indeed, graphics cards started out with the need to be able to do a multiplication per vertex plus fills of poylgons. You'll end up doing this multiplication no matter what you do! Oct 17, 2023 at 21:17

DMGregory's answer explains how this is actually handled, but I'd like to point out a couple of misconceptions:

Assume that a face of a mesh is defined by vertices A, B, C and that the normal of the face is defined by (B - A) x (C - A).

Note that this is only true for “flat shading”, where each triangle forms a visible facet, near-uniformly lit. Most meshes today have normals explicitly defined per vertex, which allows them to be interpolated to simulate curved surfaces. The normals will typically be similar to the cross product, but not equal — in particular, the 3 vertices of a triangle will have different normals, unless the surface the mesh is approximating is actually flat.

These normals cannot be calculated from the mesh, but rather are created at the same time as the mesh, and the algorithm (or author's manual operations) will depend on the intent. For example, a mesh which has the form of a hexagonal tube might be intended to be in fact hexagonal (and the normals will be perpendicular to the faces) or it might be intended to be a low-polygon approximation to a cylinder or wire (in which case the normal of each vertex will point radially outward, so faces have varying normals, and lighting is smooth across the edge between faces).

For game graphics purposes, you should understand normals as a property vertices have, which we wish to preserve the meaning of under various transformations; not as something you can derive from the mesh except when you are making the particular aesthetic restriction of using only flat shading.

“Face normals” are not used or computed at all (unless you are doing something unusual. Normals are interpolated per-pixel across the surface of a triangle, to approximate curved surfaces.

To my knowledge there is no algebraic property of the cross product that lets us simply do R((B - A) x (C - A)) to rotate the normal just like all the vertices.

Yes and no. The matrix can be such that R((B - A) x (C - A))R(B - A) x R(C - A), but in that case, it is not a rotation matrix — it is some other less restricted sort of matrix.

But, since normals are not in practice defined by cross products of vertex differences, the cross product doesn't matter. What does matter is that the vertex normals are transformed in a way which matches the transformation of the shape the mesh forms, so they remain appropriate.

In the simple case of a pure rotation matrix, which does not change the angles between vectors, it suffices to rotate the normals with the same matrix. Picture the normals as pins sticking out of the vertices, and understand that we are rotating — or more generally, performing a rigid transformation — of the entire object as a whole.

More generally — such as if the matrix contains non-uniform scale — it is necessary to use a different matrix to transform the normals (as DMGregory's answer mentions), and that is what is in fact done in general-purpose renderers.

• Thank you! Some followups (1) How are the vertex normals initialized since an individual point doesn't inherently have a direction? (2) Are face normals just barycentric interpolants of the vertex normals? Oct 17, 2023 at 4:05
• @Carpetfizz I've edited my answer to cover those points, but in brief: (1) The vertex normals are created by the same tool that created the vertex positions. What values it will choose depends on the intent — whether the continuous surface, that the mesh is a discrete approximation of, is intended to be curved or sharp. (2) "Face normals" are not part of the system; when rendering, normals are interpolated across the triangle for every pixel, just like colors/textures are. The barycenter is not a special point. Oct 17, 2023 at 5:07
• Vertex normals can be auto-calculated when we don't have a more specialized function generating them or human tweaking at play — see past Q&A about it here. To do that, we use the cross product to get a face normal for each face as a temporary value, then for each vertex, we average the normals for each face sharing that vertex, weighted by the area of that face. So only the averaged-out vertex normals survive — the face normals are just a calculation step for getting there. Oct 17, 2023 at 10:42

A few words on why people use the inverse transpose when mapping normals: suppose your normal at a point $$\p_1\$$ on the untransformed object is $$\n\$$ and $$\p_2\$$ is a nearby point on the object. Suppose also that you are applying the affine transformation matrix $$\M\$$ to your object, so that $$\Mp_1\$$ and $$\Mp_2\$$ are the new locations of $$\p_1\$$ and $$\p_2\$$ on the transformed object.

You want to transform the normal using a matrix $$\N\$$ so that the angle between $$\n\$$ and $$\p_2 - p_1\$$ is preserved:

\begin{align} (Nn) \cdot (Mp_2-Mp_1) &= n \cdot (p_2 - p_1) \\ n^T N^T M (p_2 - p_1) &= n^T (p_2-p_1) \end{align} and this equation is true for any choice of $$\p_2\$$ if $$\N^TM\$$ is the identity, i.e. $$\N = M^{-T}\$$.

• We have some previous Q&A that explains considerations for transforming normals in more detail, with diagrams. This answer may be more appropriate posted on that question. Oct 19, 2023 at 18:11
• I edited the code markdown into MathJax markdown. If you need to fix anything (for instance if I got an implied operation or grouping wrong) you can find the formatting information in our MathJax guide. Oct 23, 2023 at 3:03
• @Pikalek Thanks! I actually tried tex originally but it didn't work for some reason so I assumed it wasn't enable on this particular Stack. Oct 24, 2023 at 15:07
• You're welcome! As far as I know, we don't have full Tex support; if that's what you tried, that's probably why it didn't render as you expected. Or it's possible that you ran into some uncommon edge case error - if you are using MathJax & run into a similar problem, feel free to ask about it on our Meta or chat. Oct 24, 2023 at 15:55