Hello!
This question is about transforming normal vectors from normal maps properly. I have tried methods from over five sources, still can't get it to work, and I'm really desperate.
Included are three pictures, each one a screenshot from my app. The mesh is just a flattened cube smoothed on the edges. A blue light is always OVER the mesh.
On the first one no normal map is used, and the shader gets normals from interpolating vertex normals. As you can see, the lighting is fine and works as it should. The bottom is not lit.
On the second picture a normal map which is just blue is used - the effect should be the same as in picture one, because the normals all point upwards from the triangle. However, it's broken. The top is half-lit - it looks like the light gets cut off with a diagonal line. The part which is dark on the top is lit on the bottom, even though it should be completely dark, as the light is over the mesh.
On the third picture an actual normal map is used, and as you can see it's broken in a similar fashion to picture 2, except that some of the effects on the map are visible.
The lighting using vertex normals works properly, and the normal maps are sent properly as well, so a conclusion can be drawn that the broken part of my application is the one which either computes tangents/sends them to OpenGL/transforms the normal map using the tangent, normal and binormal vector.
I have been testing this over and over again for a month now, and just can't figure it out. Firstly, here is my function which computes the tangents:
GLvoid calculateTangents() {
for (GLuint i_index = 0; i_index < indices.size(); i_index += 3) {
// get three vertices describing a face
Vertex& v1 = vertices[indices[i_index]];
Vertex& v2 = vertices[indices[i_index + 1]];
Vertex& v3 = vertices[indices[i_index + 2]];
// calculate two edge vectors representing the triangle edges
glm::vec3 edge1 = v2.position - v1.position;
glm::vec3 edge2 = v3.position - v1.position;
// calculate the differences in U and V coordinates of the two edges above
GLfloat deltaU1 = v2.texCoords.x - v1.texCoords.x;
GLfloat deltaV1 = v2.texCoords.y - v1.texCoords.y;
GLfloat deltaU2 = v3.texCoords.x - v1.texCoords.x;
GLfloat deltaV2 = v3.texCoords.y - v1.texCoords.y;
// expanded matrix inverse
GLfloat f = 1.0f / (deltaU1 * deltaV2 - deltaU2 * deltaV1);
// the tangent vector of this face
glm::vec3 tangent(0.0f, 0.0f, 0.0f);
tangent.x = f * (deltaV2 * edge1.x - deltaV1 * edge2.x);
tangent.y = f * (deltaV2 * edge1.y - deltaV1 * edge2.y);
tangent.z = f * (deltaV2 * edge1.z - deltaV1 * edge2.z);
v1.tangent += tangent;
v2.tangent += tangent;
v3.tangent += tangent;
}
for (GLuint i_vert = 0; i_vert < vertices.size(); i_vert++) {
Vertex& vert = vertices[i_vert];
vert.tangent = glm::normalize(vert.tangent);
// Gramm-Schmidt reorthogonalization
//don't use this for simple meshes like cubes - screws 'em up
//vert.tangent = glm::normalize(vert.tangent - glm::dot(vert.tangent, vert.normal) * vert.normal);
}
}
It seems to be working fine, I've checked its compatibility with a few tutorials, including it just in case somebody spots an error.
Then, here are the vertex shader parts responsible for sending the normal, tangent and bitangent to fragment/pixel shader:
layout(location = 1) in vec3 vertNorm_model;
layout(location = 2) in vec3 vertTangent_model;
uniform mat4 modelToCameraMat4;
smooth out vec3 fragNorm_cam;
smooth out vec3 fragTangent_cam;
smooth out vec3 fragBitangent_cam;
in main():
// reorthogonalize the tangent and normal using Gramm-Schmidt
vec3 vertTangent_model_reort = normalize(vertTangent_model - dot(vertTangent_model, vertNorm_model) * vertNorm_model);
vec3 vertBitangent_model = normalize(cross(vertTangent_model_reort, vertNorm_model));
// export normal, tangent, bitangent in camera space
mat3 modelToCameraMat3 = mat3(modelToCameraMat4);
fragNorm_cam = normalize(modelToCameraMat3 * vertNorm_model);
fragTangent_cam = normalize(modelToCameraMat3 * vertTangent_model_reort);
fragBitangent_cam = normalize(modelToCameraMat3 * vertBitangent_model);
And lastly, here are the fragment/pixel shader parts doing the normal map transformations:
smooth in vec3 fragNorm_cam;
smooth in vec3 fragTangent_cam;
smooth in vec3 fragBitangent_cam;
in main():
// retrieve a normal vector from map (in tangent space) and move it from [0; 1] to [-1; 1]
vec3 fragNorm_tangent = 255.0/128.0 * texture(normalSampler, fragTexCoords).rgb - 1.0;
// transform the normal vector from tangent space to camera space
vec3 fragBumpNorm_cam = (fragNorm_tangent.x * fragTangent_cam) + (fragNorm_tangent.y * fragBitangent_cam) + (fragNorm_tangent.z * fragNorm_cam);
fragBumpNorm_cam = normalize(fragBumpNorm_cam);
// fragNormal_cam is used in lighting
vec3 fragNormal_cam = fragBumpNorm_cam;
// if instead of using the normal map vector i use the interpolated vertex normal
//vec3 fragNormal_cam = fragNorm_cam;
// i get results like in picture one - proper lighting
If anybody manages to spot the issue here, I'd be unimaginably grateful, so thanks in advance.
