How can I calculate normals using a vertex and index buffer?

Question

Is it possible to calculate normals using the positions in a vertex buffer and index buffer? Would someone show pseudocode for the algorithm?

Answer 1

Yes it is, here is how you can do:

Every vertex in the vertex buffer is represented by one index which may occur several times in the index buffer. What you first need to do is to make a function that for a index returns the corresponding vertex. Should be fairly trivial.

I am going to assume that the mesh is rendered as GL_TRIANGLES

for every 3 indexeses in index buffer
  edge1 = index1 - index2
  edge2 = index1 - index3

  var normal = edge1 cross edge2

Simple as that! This normal can either be sent once for every 3 vertices making up a face, or one time for every 3 vertices

This normal though, will be a face normal, meaning that you will get a flat looking shading, so what you might want to do is calculate the interpolated per vertex normals. What you do then is that you for every vertex keep a map that tells which face this vertex belongs to. Then for all vertexes you take all faces that this vertex belongs to, take their normals, add them together and then normalize the resulting vector. Maybe you should even weight the face normals according to size of the face

Answer 2

Another user was struggling to implement this with the existing answer, so I thought I'd show a slightly deeper code example for folks in a similar situation.

I'll use Unity C# syntax since it's what I use most often, but the same steps can be applied to any language/framework.

CalculateVertexNormals(Vector3[] vertexPositions, int[] triangleIndices, Vector3[] vertexNormals)
{

    // Zero-out our normal buffer to start from a clean slate.
    for(int vertex = 0; vertex < vertexPositions.Length; vertex++)
        vertexNormals[vertex] = Vector3.zero;

    // For each face, compute the face normal, and accumulate it into each vertex.
    for(int index = 0; index < triangleIndices.Length; index += 3) {
        int vertexA = triangleIndices[index];
        int vertexB = triangleIndices[index + 1];
        int vertexC = triangleIndices[index + 2];    

        var edgeAB = vertexPositions[vertexB] - vertexPositions[vertexA];
        var edgeAC = vertexPositions[vertexC] - vertexPositions[vertexA];

        // The cross product is perpendicular to both input vectors (normal to the plane).
        // Flip the argument order if you need the opposite winding.    
        var areaWeightedNormal = Vector3.Cross(edgeAB, edgeAC);

        // Don't normalize this vector just yet. Its magnitude is proportional to the
        // area of the triangle (times 2), so this helps ensure tiny/skinny triangles
        // don't have an outsized impact on the final normal per vertex.

        // Accumulate this cross product into each vertex normal slot.
        vertexNormals[vertexA] += areaWeightedNormal;
        vertexNormals[vertexB] += areaWeightedNormal;
        vertexNormals[vertexC] += areaWeightedNormal;
    }       

    // Finally, normalize all the sums to get a unit-length, area-weighted average.
    for(int vertex = 0; vertex < vertexPositions.Length; vertex++)    
        vertexNormals[vertex] = Vector3.Normalize(normal);
}