I'm trying to use WebGL to render some simple models, and I'm running into issues where pixels near the edge of my model are passing normals to my fragment shader that point away from the camera. This is of course messing with my attempts to use physically-based shading, because BRDFs are only defined in the hemisphere around the normal vector. These normals aren't just slightly pointing away from the screen; if that was the case, I would chalk it up to floating-point error and clamp to acceptable values. But some of my pixels have v-dot-n values of -0.5 or less.
This happens on pretty much any mesh I try to render. I have verified that my vertex normals point away from the mesh. I have backface culling turned on. This happens with both perspective and orthographic projections. I have tested in both Firefox and Chrome. Everything else about my render looks fine.
My vertex shader looks like this:
precision mediump float;
uniform mat4 M;
uniform mat4 V;
uniform mat4 P;
attribute vec3 vertexPos_model;
attribute vec3 vertexNormal_model;
varying vec3 pos_world;
varying vec3 normal_world;
void main() {
gl_Position = P * V * M * vec4(vertexPos_model, 1.0);
pos_world = (M * vec4(vertexPos_model, 1.0)).xyz;
// We can use M here instead of its inverse transpose because it does not scale the model.
normal_world = mat3(M) * vertexNormal_model;
}
And my fragment shader looks like this:
precision mediump float;
uniform vec3 cameraPos_world;
varying vec3 pos_world;
varying vec3 normal_world;
void main() {
vec3 n = normalize(normal_world);
vec3 v = normalize(cameraPos_world - pos_world);
float vAngleCos = dot(n, v);
// This will highlight pixels whose view vectors are outside of the hemisphere around n.
gl_FragColor = vec4(vec3(clamp(-vAngleCos, 0.0, 1.0)), 1.0);
}
Rendering with the fragment shader above gives me results like this:
Apologies if I'm missing something obvious!
