GLSL - multiplying fragment output color by normal after calling texture function displays wrong texture

Question

So I'm quite new to OpenGL/GLSL. I'm trying to make multitexturing and simple normal lighting work. Here is a Vertex shader:

#version 130

uniform mat4 transform;

in vec3 vertPos;
in vec2 texUV;
in vec3 vertNorm;
in uint thisLayer;

out vec2 fragTexCoord;
out vec3 fragNorm;
out float fragLayer;

void main()
{
    gl_Position = transform * vec4(vertPos, 1.0f);

    fragTexCoord = texUV;
    fragLayer = float(thisLayer);
    fragNorm = vertNorm;
}

... and here is Fragment shader:

#version 130

uniform sampler2DArray texArray;
uniform uint texCount;
uniform float rTime;

in vec2 fragTexCoord;
in vec3 fragNorm;
in float fragLayer;

out vec4 color;

void main()
{
    float actual_layer = max(0, min(texCount - uint(1), floor(fragLayer + 0.5))); // Getting texture layer
    vec3 texCoord = vec3(fragTexCoord, actual_layer);

    color = texture(texArray, texCoord);
}

And it works fine:

Multiplying color.rbg by any constant works as expected:

color = texture(texArray, texCoord);
color.rgb *= vec3(0.5, 0.5, 0.5);

But if I try to multiply color.rgb by a normal, like this:

vec3 finalNorm = (normalize(fragNorm) + vec3(1,1,1))/vec3(2,2,2);
color = texture(texArray, texCoord);
color.rgb *= finalNorm;

The wall texture changes to the last texture in the array (the one on the top):

I have no idea why this happens. I normalize vertex normals and map their values into [0, 1] range, so colors will stay in [0, 1] range after multiplication as well. It is supposed to simply shift the colors intensity, like it did while multiplying by a constant, not change the texture itself, right? I am aware of the fact that this is not how you perform normal lighting, however, this is not the correct behavior as well, and I'm ripping my hair off as to why. Any help is appreciated. I will provide more info if needed.

EDIT: OS is Windows 10. GPU's: Intel HD 4600/Nvidia GTX 860M (happens on both of them). OpenGL version 4.3.0 build 20.19.15.4624.

Some additional info:

I use SOIL library to load data from PNG files. I use:

glTexImage3D(GL_TEXTURE_2D_ARRAY, 0, GL_RGBA, img_width, img_height, layer_count * 2, 0, GL_RGBA, GL_UNSIGNED_BYTE, 0);

to create a container for texture array, and use:

glTexSubImage3D(GL_TEXTURE_2D_ARRAY, 0, 0, 0, i, img_width, img_height, layer_count, GL_RGBA, GL_UNSIGNED_BYTE, data);

to load data into it. One issue I have with this is that glTexImage3D takes layer_count (the number of textures) multiplied by 2, and I have no idea why as well! Maybe this is somehow related to the issue I'm having above?

Answer 1

We need information on OS, GPU, drivers and opengl version you are using.

This appears like it may be the result of optimization after undefined behavior. To debug try the following:

#version 130

uniform sampler2DArray texArray;
uniform uint texCount;
uniform float rTime;

in vec2 fragTexCoord;
in vec3 fragNorm;
in float fragLayer;

out vec4 color;

void main()
{
    float actual_layer = max(0, min(texCount - uint(1), floor(fragLayer + 0.5))); // Getting texture layer
    vec3 texCoord = vec3(fragTexCoord, actual_layer);

    vec3 finalNorm = (normalize(fragNorm) + vec3(1,1,1))/vec3(2,2,2);
    color = texture(texArray, texCoord);
    color.rgb *= finalNorm;

}

Make sure this is the exact information for you fragment shader and make sure what you posted is the exact text for your vertex shader. If either of these aren't true, copy and paste the code and now try again to see if you have the same issue.

Next, you aren't calculating the normal correct, if you have the correct vertex normals in the first place, there is no need to A: normalize, they should already be normalized, and B: multiply and divide by a similar vector. I see you are trying to change the range of the normal from -1 +1 to 0, 1 with this.

First, make sure your normals are pre-normalized already. Next make sure they are correct, 0,0,0 is not a valid value for a normal, as the normalization for this may be undefined depending on your opengl spec, which can cause erratic behavior in your program from glsl compiler assumptions.

Next change your normal calculation to be the following:

vec3 finalNorm = (normalize(fragNorm) + 1.0)/2.0;

adding a scalar will add to all values of a vector, division by scalar will divide all values in the vector. normalization of something that is already normalized should not be an issue.

if things still haven't been sorted out, try manually multiplying color.rgb by an incorrect value (via forcing incorrect values on your vertex attributes)

color.rgb *= 1.0/0.0;

if you still see the same behavior you know there is some bad numeric value propagation happening. The above should appear as all white (except background).

color.rgb *= -1.0/0.0;

The above should appear as all black (except background).

color.rgb *= 0.0;

The above should be all black as well (except background).

Some more sanity checks, stop using uint values you may incidentally be causing undefined behavior in an un-obvious way. Try the following

float actual_layer = max(0, min(texCount - 1, floor(fragLayer + 0.5)));

You may also benefit from moving your camera positioning in relation to your object, seeing if you see any additional artifacts as hints.

Finally try changing the version on your shader to a more modern version of GLSL, such as #version 330 if you are using opengl 3.3 and above.

If none of these things helped, you are either on a system with actual driver bugs, in which case update your drivers/change system. If it still has issues you more than likely have an issue in your code base that we can't help you with because you aren't showing us what you are doing out side of your shaders.