I'm playing with some terrain generation. I have a triangular mesh that looks OK - that is if I hardcode verticles normals (on this image every single verticle has normal of (0,-1,0) Island

The situation is different when I try to calculate normals normally. I.E.

// for every vertex in triangle normal is
private Vector3D GetNormal(Vector3D a, Vector3D b, Vector3D c)
    //return new Vector3D(0, -1, 0);
    return (c - b).CrossProduct(a - b).Normalize();

In this case I end up with Island with missing triangles

Is something fundamentally wrong with my approach ? I've read that vertex normals should take into account adjected triangles (sum of adjected triangles normals normalized by area those triangles have), but I don't think my "simplistic" approach should create those weird holes. Sources (if anyone is interested: https://github.com/svejdo1/Delaunay)

  • 1
    \$\begingroup\$ I think something is wrong with the triangle winding. \$\endgroup\$ – ratchet freak Feb 14 '17 at 16:02

It looks like some normals are pointing the wrong way. This means that those triangles aren't wound correctly.

As a quick fix you can do:

Vec3 norm = (c - b).CrossProduct(a - b).Normalize();
if(norm.y < 0) //or whatever direction up is
    norm = -norm; 
return norm;

A better fix would be to debug your triangulation code and make sure that the order of vertices are consistent.

  • \$\begingroup\$ A robust addition in certain situations, if the geometry is complex (e.g. there are triangles facing down in the landscape e.g. in a cave) but doesn't have sharp edges (e.g. adjacent triangles more than 90 degrees off), is to flip the normal only if it is facing the opposite direction of an adjacent triangle's normal (dot product < 0). Then you end up with all of the normals facing the same direction, and after that you can make a guess based on the situation, e.g. count how many face "down" and if it's the majority, flip them all. But yeah fixing the triangulation code is a much better fix. \$\endgroup\$ – Jason C Feb 14 '17 at 17:42
  • \$\begingroup\$ I'm marking this as answer since this IS the way how to calculate normal over complex mesh. My problem was in the end with specular shader. Moral from my story is if you don't really understand topic (shaders) start with simple cases and move to more complex - don't copy-paste complex and assume it will just work :) \$\endgroup\$ – Ondrej Svejdar Mar 5 '17 at 21:52

There's an error in your maths. You should use the cross product between two vectors with the same start position, but one of your vectors goes from b to c, the other goes from a to b. You need to change the line to

return (c - b).CrossProduct(a - b).Normalize();

If you're still having trouble, then normalize the 2 vectors before using the cross product.

  • \$\begingroup\$ Thanks, I played with it for long I pasted wrong version :) Anyway the holes are in even for corrected vertex diffs. \$\endgroup\$ – Ondrej Svejdar Feb 14 '17 at 15:33
  • \$\begingroup\$ Well I guess your answer no longer makes sense... \$\endgroup\$ – Vaillancourt Feb 14 '17 at 15:33
  • \$\begingroup\$ @Ondrej read the edit \$\endgroup\$ – Bálint Feb 14 '17 at 15:48
  • \$\begingroup\$ @Alexandre deleting an answer has a negative effect on my account, so this stays here. \$\endgroup\$ – Bálint Feb 14 '17 at 15:48
  • \$\begingroup\$ Normalizing the two vectors before computing the cross product won't make a difference. a.CrossProduct(b).Normalize() is mathematically identical to (and also more performant and potentially more robust wrt precision problems than) a.Normalize().CrossProduct(b.Normalize()). And something like a.Normalize().CrossProduct(b.Normalize()).Normalize() is entirely unnecessary. PS Beware degenerate triangles with edge cross products of magnitude 0, which shouldn't exist in a proper surface but if the possibility is there, handle it appropriately. \$\endgroup\$ – Jason C Feb 14 '17 at 17:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.