4
\$\begingroup\$

I have a working open gl es 2.0 3d engine (my own) that I've used in several iOS apps. I recently noticed that on a mutli-symmetrical object the lighting didn't look the same at each rotation. For example, I would think this 6 sided object's illumination (think throwing star) should appear exactly the same at 0 deg and 60, 120, 180, 240 and 300. Apparently in my other games I haven't done the same types of rotations that I'm doing in this app. So I never noticed this issue.

Digging deeper and trying to debug the issue I found two symptoms. Each might have it's own source problem.

For testing I created a solid color polyhedron. I also simplified the shader and removed shadow mapping and set the normal map to 0,0,1 (in the texture 127,127,255). Also in the code below I'm only displaying the diffuse component to try to debug things, but have included my full shading code.

Problem 1 is that when the object rotates the lighting changes. In the screenshot the light and the camera are right above the object. Therefore at 0 deg and 180 deg the illumination on each face should appear the same. But 180 deg is brighter. In between it ramps.

FYI: The shadow in all these screenshots is a representation of where the light is coming from. The shadow map is being rendered on the paper only.

Problem 1

Problem 2 is that when the camera is moved the light appears to move with the camera. After some testing I know why this one is happening (below).

Problem 2

Here's all the code. First the vertex shader:

precision highp float;

uniform mat4 u_modelViewProjectionMatrix;
uniform vec3 u_lightPosition;
uniform mat3 u_normalMatrix;
uniform mat4 u_matrix2;

uniform vec3 u_cameraPosition;

attribute vec3 a_vertexPosition;
attribute lowp vec2 a_vertexTexCoord0;
attribute lowp vec3 a_vertexNormal;
attribute lowp vec3 a_vertexTangent;

varying lowp vec2 v_fragmentTexCoord0;
varying lowp vec2 v_fragmentTexCoord3;

varying vec3 v_eyeVec;
varying vec3 v_lightVec;
varying vec3 v_position;

void main(void)
{
  v_fragmentTexCoord0 = a_vertexTexCoord0;
  v_fragmentTexCoord3 = a_vertexTexCoord0 - vec2(0.0, 0.5);  // normal is down half

  gl_Position = u_modelViewProjectionMatrix * vec4(a_vertexPosition,1.0);

  lowp vec3 normal = normalize(u_normalMatrix * a_vertexNormal);
  lowp vec3 tangent = normalize(u_normalMatrix * a_vertexTangent);
  lowp vec3 bitangent = cross(normal,tangent);
  mat3 TBNMatrix = mat3(tangent, bitangent, normal);

  //mat3 TBNM2 = mat3(a_vertexTangent, cross(a_vertexNormal,a_vertexTangent), a_vertexNormal);

  //----------------- Compute Eye vector ---------------
  v_eyeVec =  u_cameraPosition - a_vertexPosition.xyz;
  v_eyeVec *= TBNMatrix;

  //----------------- Compute Light vector ---------------
  v_lightVec = u_lightPosition - a_vertexPosition.xyz;
  v_lightVec *= TBNMatrix;
}

Fragment Shader:

precision highp float;

uniform sampler2D u_textureSampler;
uniform sampler2D u_textureSampler2;

uniform lowp vec4 u_lightColorAmbient;
uniform lowp vec4 u_lightColorDiffuse;
uniform lowp vec4 u_lightColorSpecular;     // the specular is now a map
uniform float u_shininess;

varying vec3 v_eyeVec;
varying vec3 v_lightVec;
varying lowp vec2 v_fragmentTexCoord0;
varying lowp vec2 v_fragmentTexCoord3;

void main(void)
{

  lowp vec4 baseColor = texture2D(u_textureSampler,v_fragmentTexCoord0);
  lowp vec4 normColor = texture2D(u_textureSampler, v_fragmentTexCoord3);
  lowp vec3 norm = normColor.rgb * 2.0 - 1.0;

  // how shiny is: specular map range (0-1) represents 100 through u_shininess. 1.0 = u_shininess, 0.0 = 100...
  float specularMap = 100.0 - (100.0-u_shininess)*normColor.a;
  // baseColor.a is the specularFactor of how much shininess to apply, to dull things down

  vec3 lightVector = normalize(v_lightVec);
  float lamberFactor = max(0.0, dot(norm, lightVector));

  vec3 diffuseComponent  = vec3(0.0,0.0,0.0);
  vec3 specularComponent = vec3(0.0,0.0,0.0);

  if (lamberFactor > 0.0)
  {
     vec3 halfVector = normalize(v_lightVec + v_eyeVec);
     float specularPower = min(1.0, max(0.0,pow(dot(norm, halfVector),specularMap)));  //baseColor.a is a spec factor texture

     diffuseComponent  = u_lightColorDiffuse.rgb  * baseColor.rgb * lamberFactor;
     specularComponent = u_lightColorSpecular.rgb * baseColor.a * specularPower;
  }

  vec3 ambientComponent = u_lightColorAmbient.rgb  * baseColor.rgb;

  // normally I would do
  // gl_FragColor = vec4(ambientComponent + (diffuseComponent + specularComponent) ,1.0);

  gl_FragColor = vec4(diffuseComponent ,1.0);
}

iOS code is passing these uniforms:

u_lightPosition = vec3Make(lightPos.x-thisItem->currentPos.x, lightPos.y-thisItem->currentPos.y, lightPos.z-thisItem->currentPos.z);
u_cameraPosition = vec3Make(cameraPos.x-thisItem->currentPos.x, cameraPos.y-thisItem->currentPos.y, cameraPos.z-thisItem->currentPos.z);

u_normalMatrix = getMat3FromMat4(mModelView);

Tests

Problem 2 hypothesis - the camera moving shouldn't effect the lighting but it does because the normal is being multiplied by the normal matrix. I understood that the normal matrix is the same as the modelView when using uniform scaling. But the camera move is in the view matrix. So by replacing the above with:

u_normalMatrix = getMat3FromMat4(thisItem->objMatrix);

..the light no longer moves with the camera and that problem appears to go away. But that's not what I've read is the right way to do it. And it doesn't solve problem #1. It also seems like the specular light would need the modelView to be right and not just the model matrix.

UPDATE: To clarify things here a little more regarding coordinate systems. All objects, the camera, and the light are stored in world coordinates. The Z axis has numbers getting smaller as they go away from camera. In this particular case the camera is at 160.0 and the paper is at 100.2861 which is a calculated depth for a 1:1 pixel unity. The light is about 131, between the camera and the paper. The object is about 8 units in diameter.

When I draw objects, currently the object matrix starts at identity and gets scaled, rotated and transformed to it's world coordinate location. The mView is created using a lookAtMatrix(cameraPos, camLookAt, camUpVector, mView); function. which creates an inverted matrix of it's position and location in world coordinates. As you can see above the light position is provided to the shader in model space but without regard to the camera position or rotation. The camera location is also provided to the shader in model space without regard to it's rotation.

I could move the camera to 0,0,0 and in this app because of the setup math everything would automatically push farther away to appear the same. I don't know if it's better to always have the camera there or not.

In all my other games the camera is always parallel to the scene, and in this app it will be also. But I want to solve this so I have a working system for moving and rotating the camera anywhere.

Regarding problem #1. It's almost as if the normals are getting bent weird by the TBN when the object rotates. That's the more important of the two issues for me at the moment.

UPDATE 2: The test was a clue but wasn't correct. Rather than removing the view matrix from the normal matrix I needed to add it to the light position to get it in camera/eye space. That way when the camera rotates it gets applied to the light. Then as the answer suggests, I also needed to use modelView in the shader to get a version of the vertex in that space as well. Then with the vertex, camera, and light in eye space I calculate the eye and light vectors.

At the moment I'm still having issue #1 above where I get different lighting results on a rotated symmetrical object. Help on that issue is still appreciated.

FINAL UPDATE: After much testing with setting gl_FragColor to various variables. I finally figured out that it was because I was normalizing the light vector before getting the dot product to determine the lamber factor. So here this:

  vec3 lightVector = normalize(v_lightVec);
  float lamberFactor = max(0.0, dot(norm, lightVector));

needed to be replaced with just:

  float lamberFactor = max(0.0, dot(norm, v_lightVec));

That solved it.

\$\endgroup\$
  • \$\begingroup\$ when you say thisItem->objMatrix what does this matrix do? I assume it transforms from local->world. And what does mModelView do? I assume it transforms from world->eye. I assume your problem is space inconsistency, but it's hard to debug the code since it's not obvious in what spaces each calc are done. I have a habbit of prefixing the vec name with the space. \$\endgroup\$ – concept3d Dec 16 '14 at 10:46
  • \$\begingroup\$ All positions start out in world coordinates and many are shown on the screen. The objMatrix transforms from model space (identity) to world. mView is a lookAt from the camera in world coordinates. mModelView is the product of those two. If necessary I can post al the transformation code or detail the coordinate system. \$\endgroup\$ – badweasel Dec 16 '14 at 12:01
  • \$\begingroup\$ The light and camera positions are converted to the moved model's space before being sent to the shader by subtraction. \$\endgroup\$ – badweasel Dec 16 '14 at 17:16
  • \$\begingroup\$ To clarify, the -> is a 'c' syntax pointing to the member of a struct. So that's the item's property. I didn't intend your notion of it meaning transforming from one space to another. It's literally a copy paste of my code. \$\endgroup\$ – badweasel Dec 16 '14 at 17:20
  • \$\begingroup\$ "I learned that the normal matrix for orthogonal the same as the modelView." To be more precise, the normal matrix is not necessarily the same as the top-left 3x3 matrix (rotation) of the ModelView matrix - if you have non-uniform scaling in your ModelView matrix then you need to take the inverse transpose. That is actually a somewhat uncommon situation, but in order to cover all bases, the formal definition of the Normal Matrix does just that: NormalMatrix = transpose (inverse (ModelViewMatrix)). \$\endgroup\$ – Andon M. Coleman Dec 16 '14 at 21:35
5
\$\begingroup\$

From what I read from the shaders the light is in world-space and the light calculation is done on the object in part in untransformed object-space.

You need to compute your lighting with both light & model in world-space or both in camera space.

Whenever moving the camera messes up the lighting it means some of the data is calculated in a different space from the rest.

(a_vertexPosition.xyz) 

is in object space (before the object is placed in the 3D world).

(a_vertexPosition.xyz * modelmatrix) 

is in world space.

((a_vertexPosition.xyz * modelmatrix) * viewmatrix) 

is in view (camera) space.

(((a_vertexPosition.xyz * modelmatrix) * viewmatrix) * projectionmatrix) 

is in screen space (just before perspective).

vec4 temp = ((a_vertexPosition.xyz * modelmatrix) * viewmatrix) * projectionmatrix;
temp /= temp.w; 

is in projected screen space. The GPU does this division by w just before the pixel shader. Sometimes you need to calculate this in the vertex shader to do 2D effects.

To calculate the lighting properly you probably need to use (a_vertexPosition.xyz * modelmatrix) instead of plain a_vertexPosition.xyz for calculating the light and camera vectors.

\$\endgroup\$
  • \$\begingroup\$ I think my issue has been not understanding how to correctly get things in the proper space, and what space certain calculations need to be in. I've been lucky previously because, say, if I didn't rotate the camera or if the camera was at 0,0,0 I didn't see certain problems. Tonight I'm working with an even simpler model and thinking through the best way to get things in the right space. For example I was just thinking that my light needs to be multiplied by the view matrix to put it in the same eye space that the model is put in. Then calculate directional vectors from that. \$\endgroup\$ – badweasel Dec 23 '14 at 0:16
  • \$\begingroup\$ And the camera in eye space should be 0,0,0. I also get fuzzy when starting to think about rotating the camera and how that effects relative light positions and therefore directional vectors. \$\endgroup\$ – badweasel Dec 23 '14 at 0:19
  • \$\begingroup\$ You can put both in view space, or world space, it'll work so long as they are consistent. I like having it all in world space, not as efficient but I find it easier to work with. \$\endgroup\$ – Stephane Hockenhull Dec 23 '14 at 0:21
  • \$\begingroup\$ Right but part of my issue is how to put them in the right space and which space my shaders are expecting them to be in. Also assuming that lightPosition, cameraPosition, and modelPosition are all starting off in world space and none of them are at 0,0,0 and that the camera and model can both be rotated. \$\endgroup\$ – badweasel Dec 23 '14 at 1:37
  • \$\begingroup\$ Added the different spaces to the answer (comments don't allow formatting) \$\endgroup\$ – Stephane Hockenhull Dec 23 '14 at 8:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.