OpenGL light appears to move with camera and changes with object rotation

Question

I have a working open gl es 2.0 3d engine (my own) that I've used in several iOS apps. I recently noticed that on a mutli-symmetrical object the lighting didn't look the same at each rotation. For example, I would think this 6 sided object's illumination (think throwing star) should appear exactly the same at 0 deg and 60, 120, 180, 240 and 300. Apparently in my other games I haven't done the same types of rotations that I'm doing in this app. So I never noticed this issue.

Digging deeper and trying to debug the issue I found two symptoms. Each might have it's own source problem.

For testing I created a solid color polyhedron. I also simplified the shader and removed shadow mapping and set the normal map to 0,0,1 (in the texture 127,127,255). Also in the code below I'm only displaying the diffuse component to try to debug things, but have included my full shading code.

Problem 1 is that when the object rotates the lighting changes. In the screenshot the light and the camera are right above the object. Therefore at 0 deg and 180 deg the illumination on each face should appear the same. But 180 deg is brighter. In between it ramps.

FYI: The shadow in all these screenshots is a representation of where the light is coming from. The shadow map is being rendered on the paper only.

Problem 2 is that when the camera is moved the light appears to move with the camera. After some testing I know why this one is happening (below).

Here's all the code. First the vertex shader:

precision highp float;

uniform mat4 u_modelViewProjectionMatrix;
uniform vec3 u_lightPosition;
uniform mat3 u_normalMatrix;
uniform mat4 u_matrix2;

uniform vec3 u_cameraPosition;

attribute vec3 a_vertexPosition;
attribute lowp vec2 a_vertexTexCoord0;
attribute lowp vec3 a_vertexNormal;
attribute lowp vec3 a_vertexTangent;

varying lowp vec2 v_fragmentTexCoord0;
varying lowp vec2 v_fragmentTexCoord3;

varying vec3 v_eyeVec;
varying vec3 v_lightVec;
varying vec3 v_position;

void main(void)
{
  v_fragmentTexCoord0 = a_vertexTexCoord0;
  v_fragmentTexCoord3 = a_vertexTexCoord0 - vec2(0.0, 0.5);  // normal is down half

  gl_Position = u_modelViewProjectionMatrix * vec4(a_vertexPosition,1.0);

  lowp vec3 normal = normalize(u_normalMatrix * a_vertexNormal);
  lowp vec3 tangent = normalize(u_normalMatrix * a_vertexTangent);
  lowp vec3 bitangent = cross(normal,tangent);
  mat3 TBNMatrix = mat3(tangent, bitangent, normal);

  //mat3 TBNM2 = mat3(a_vertexTangent, cross(a_vertexNormal,a_vertexTangent), a_vertexNormal);

  //----------------- Compute Eye vector ---------------
  v_eyeVec =  u_cameraPosition - a_vertexPosition.xyz;
  v_eyeVec *= TBNMatrix;

  //----------------- Compute Light vector ---------------
  v_lightVec = u_lightPosition - a_vertexPosition.xyz;
  v_lightVec *= TBNMatrix;
}

Fragment Shader:

precision highp float;

uniform sampler2D u_textureSampler;
uniform sampler2D u_textureSampler2;

uniform lowp vec4 u_lightColorAmbient;
uniform lowp vec4 u_lightColorDiffuse;
uniform lowp vec4 u_lightColorSpecular;     // the specular is now a map
uniform float u_shininess;

varying vec3 v_eyeVec;
varying vec3 v_lightVec;
varying lowp vec2 v_fragmentTexCoord0;
varying lowp vec2 v_fragmentTexCoord3;

void main(void)
{

  lowp vec4 baseColor = texture2D(u_textureSampler,v_fragmentTexCoord0);
  lowp vec4 normColor = texture2D(u_textureSampler, v_fragmentTexCoord3);
  lowp vec3 norm = normColor.rgb * 2.0 - 1.0;

  // how shiny is: specular map range (0-1) represents 100 through u_shininess. 1.0 = u_shininess, 0.0 = 100...
  float specularMap = 100.0 - (100.0-u_shininess)*normColor.a;
  // baseColor.a is the specularFactor of how much shininess to apply, to dull things down

  vec3 lightVector = normalize(v_lightVec);
  float lamberFactor = max(0.0, dot(norm, lightVector));

  vec3 diffuseComponent  = vec3(0.0,0.0,0.0);
  vec3 specularComponent = vec3(0.0,0.0,0.0);

  if (lamberFactor > 0.0)
  {
     vec3 halfVector = normalize(v_lightVec + v_eyeVec);
     float specularPower = min(1.0, max(0.0,pow(dot(norm, halfVector),specularMap)));  //baseColor.a is a spec factor texture

     diffuseComponent  = u_lightColorDiffuse.rgb  * baseColor.rgb * lamberFactor;
     specularComponent = u_lightColorSpecular.rgb * baseColor.a * specularPower;
  }

  vec3 ambientComponent = u_lightColorAmbient.rgb  * baseColor.rgb;

  // normally I would do
  // gl_FragColor = vec4(ambientComponent + (diffuseComponent + specularComponent) ,1.0);

  gl_FragColor = vec4(diffuseComponent ,1.0);
}

iOS code is passing these uniforms:

u_lightPosition = vec3Make(lightPos.x-thisItem->currentPos.x, lightPos.y-thisItem->currentPos.y, lightPos.z-thisItem->currentPos.z);
u_cameraPosition = vec3Make(cameraPos.x-thisItem->currentPos.x, cameraPos.y-thisItem->currentPos.y, cameraPos.z-thisItem->currentPos.z);

u_normalMatrix = getMat3FromMat4(mModelView);

Tests

Problem 2 hypothesis - the camera moving shouldn't effect the lighting but it does because the normal is being multiplied by the normal matrix. I understood that the normal matrix is the same as the modelView when using uniform scaling. But the camera move is in the view matrix. So by replacing the above with:

u_normalMatrix = getMat3FromMat4(thisItem->objMatrix);

..the light no longer moves with the camera and that problem appears to go away. But that's not what I've read is the right way to do it. And it doesn't solve problem #1. It also seems like the specular light would need the modelView to be right and not just the model matrix.

UPDATE: To clarify things here a little more regarding coordinate systems. All objects, the camera, and the light are stored in world coordinates. The Z axis has numbers getting smaller as they go away from camera. In this particular case the camera is at 160.0 and the paper is at 100.2861 which is a calculated depth for a 1:1 pixel unity. The light is about 131, between the camera and the paper. The object is about 8 units in diameter.

When I draw objects, currently the object matrix starts at identity and gets scaled, rotated and transformed to it's world coordinate location. The mView is created using a lookAtMatrix(cameraPos, camLookAt, camUpVector, mView); function. which creates an inverted matrix of it's position and location in world coordinates. As you can see above the light position is provided to the shader in model space but without regard to the camera position or rotation. The camera location is also provided to the shader in model space without regard to it's rotation.

I could move the camera to 0,0,0 and in this app because of the setup math everything would automatically push farther away to appear the same. I don't know if it's better to always have the camera there or not.

In all my other games the camera is always parallel to the scene, and in this app it will be also. But I want to solve this so I have a working system for moving and rotating the camera anywhere.

Regarding problem #1. It's almost as if the normals are getting bent weird by the TBN when the object rotates. That's the more important of the two issues for me at the moment.

UPDATE 2: The test was a clue but wasn't correct. Rather than removing the view matrix from the normal matrix I needed to add it to the light position to get it in camera/eye space. That way when the camera rotates it gets applied to the light. Then as the answer suggests, I also needed to use modelView in the shader to get a version of the vertex in that space as well. Then with the vertex, camera, and light in eye space I calculate the eye and light vectors.

At the moment I'm still having issue #1 above where I get different lighting results on a rotated symmetrical object. Help on that issue is still appreciated.

FINAL UPDATE: After much testing with setting gl_FragColor to various variables. I finally figured out that it was because I was normalizing the light vector before getting the dot product to determine the lamber factor. So here this:

  vec3 lightVector = normalize(v_lightVec);
  float lamberFactor = max(0.0, dot(norm, lightVector));

needed to be replaced with just:

  float lamberFactor = max(0.0, dot(norm, v_lightVec));

That solved it.

Answer 1

From what I read from the shaders the light is in world-space and the light calculation is done on the object in part in untransformed object-space.

You need to compute your lighting with both light & model in world-space or both in camera space.

Whenever moving the camera messes up the lighting it means some of the data is calculated in a different space from the rest.

(a_vertexPosition.xyz) 

is in object space (before the object is placed in the 3D world).

(a_vertexPosition.xyz * modelmatrix) 

is in world space.

((a_vertexPosition.xyz * modelmatrix) * viewmatrix) 

is in view (camera) space.

(((a_vertexPosition.xyz * modelmatrix) * viewmatrix) * projectionmatrix) 

is in screen space (just before perspective).

vec4 temp = ((a_vertexPosition.xyz * modelmatrix) * viewmatrix) * projectionmatrix;
temp /= temp.w; 

is in projected screen space. The GPU does this division by w just before the pixel shader. Sometimes you need to calculate this in the vertex shader to do 2D effects.

To calculate the lighting properly you probably need to use (a_vertexPosition.xyz * modelmatrix) instead of plain a_vertexPosition.xyz for calculating the light and camera vectors.