I've just implemented a simple algorithm to calculate terrain vertex normals and I'm using diffuse and ambient light currently, but for some reason the squares that make up my terrain are visible when I turn on lighting as you can see in the above image. I'm currently calculating the normal for the terrain using the cross product formula between two vectors of the squares that compose the terrain. Is it because I should be using something like the average between the neighbour normals? I'll leave bellow the algorithm I'm using:
//m_Polygon->Position is the 3D position of the vertex.
XMVECTOR Vector1 = XMVectorSubtract(XMLoadFloat3(&m_Polygon[m_Polygon.size() - 4].Position), XMLoadFloat3(&m_Polygon[m_Polygon.size() - 3].Position)); // .--
XMVECTOR Vector2 = XMVectorSubtract(XMLoadFloat3(&m_Polygon[m_Polygon.size() - 4].Position), XMLoadFloat3(&m_Polygon[m_Polygon.size() - 1].Position)); // |
XMVECTOR Normal = XMVector3Normalize(XMVector3Cross(Vector1, Vector2));
m_Polygon[m_Polygon.size() - 4].Normal = XMFLOAT3(XMVectorGetX(Normal), XMVectorGetY(Normal), XMVectorGetZ(Normal));
Vector1 = XMVectorSubtract(XMLoadFloat3(&m_Polygon[m_Polygon.size() - 3].Position), XMLoadFloat3(&m_Polygon[m_Polygon.size() - 2].Position)); // --.
Vector2 = XMVectorSubtract(XMLoadFloat3(&m_Polygon[m_Polygon.size() - 3].Position), XMLoadFloat3(&m_Polygon[m_Polygon.size() - 4].Position)); // |
Normal = XMVector3Normalize(XMVector3Cross(Vector1, Vector2));
m_Polygon[m_Polygon.size() - 3].Normal = XMFLOAT3(XMVectorGetX(Normal), XMVectorGetY(Normal), XMVectorGetZ(Normal));
Vector1 = XMVectorSubtract(XMLoadFloat3(&m_Polygon[m_Polygon.size() - 2].Position), XMLoadFloat3(&m_Polygon[m_Polygon.size() - 1].Position)); // |
Vector2 = XMVectorSubtract(XMLoadFloat3(&m_Polygon[m_Polygon.size() - 2].Position), XMLoadFloat3(&m_Polygon[m_Polygon.size() - 3].Position)); // --.
Normal = XMVector3Normalize(XMVector3Cross(Vector1, Vector2));
m_Polygon[m_Polygon.size() - 2].Normal = XMFLOAT3(XMVectorGetX(Normal), XMVectorGetY(Normal), XMVectorGetZ(Normal));
Vector1 = XMVectorSubtract(XMLoadFloat3(&m_Polygon[m_Polygon.size() - 1].Position), XMLoadFloat3(&m_Polygon[m_Polygon.size() - 4].Position)); // |
Vector2 = XMVectorSubtract(XMLoadFloat3(&m_Polygon[m_Polygon.size() - 1].Position), XMLoadFloat3(&m_Polygon[m_Polygon.size() - 2].Position)); // .--
Normal = XMVector3Normalize(XMVector3Cross(Vector1, Vector2));
m_Polygon[m_Polygon.size() - 1].Normal = XMFLOAT3(XMVectorGetX(Normal), XMVectorGetY(Normal), XMVectorGetZ(Normal));
As you can see I'm using DirectXMath Library to calculate everything, I just calculate the normals for the last square(4 vertices distributed in clockwise order).