Transforming normal vector in GLSL

Question

I'm looking at a basic diffuse light shader here http://en.wikibooks.org/wiki/GLSL_Programming/Unity/Diffuse_Reflection.

Diffuse light needs the normal vector in eye-space, so that it can dot it with the light position (also in eye-space), and figure out the diffuse contribution. (eye-space not world-space is used, because that simplifies calculating specular contributions).

The code calculates the normal vector in the following way:

        mat4 modelMatrixInverse = _World2Object; // unity_Scale.w
           // is unnecessary because we normalize vectors

        vec3 normalDirection = normalize(
           vec3(vec4(gl_Normal, 0.0) * modelMatrixInverse));

A linear algebra calculation shows that given a matrix M, Transpose(Inverse(M)) transforms normals. The Wikipedia on normals page contains a derivation of the required formula: http://en.wikipedia.org/wiki/Normal_%28geometry%29#Transforming_normals (I've actually rewritten that section myself, as it was not lucid).

So does this mean that the 'vector * matrix' overload in GLSL is implemented as:

    Transpose(vector) * Matrix

just holding a row-vector instead of a column-vector in the output, i.e.:

    Transpose( Transpose(vector) * matrix )

or more simply:

    Transpose(matrix) * vector

?

---

I've also found what appears to be alternative: gl_NormalMatrix

But in lines 9-11 here http://pixelsorcery.wordpress.com/2010/07/13/fisheye-vertex-shader/ ) I see the following code:

normal = vec4(gl_NormalMatrix * gl_Normal, 0.0);
vec4 vVertex = gl_ModelViewMatrix * gl_Vertex;
light_dir = gl_LightSource[0].position - vVertex;

Would I be right to assume that gl_NormalMatrix gives a normal in eye-space, and gl_LightSource[0].position gives the light location also in eye-space?

http://www.opengl.org/discussion_boards/showthread.php/172231-gl_NormalMatrix-Replacement suggests gl_NormalMatrix is deprecated.

Does that mean the first implementation is the preferred approach?

EDIT: http://www.opengl-tutorial.org/beginners-tutorials/tutorial-8-basic-shading/#The_Diffuse_part

Answer 1

gl_NormalMatrix is deprecated, but there was never anything special about it in the first place.

It is just the inverse transpose of the upper-left 3x3 part of your ModelView matrix. This extra bit of math effectively gets rid of any issues caused by non-uniform scaling and gets rid of translation by removing the 4th column of the ModelView matrix.

Rather than doing this operation in your vertex shader for each vertex, if you need to transform normals you should pre-compute the normal matrix on the CPU whenever you update your ModelView matrix. That was what GL did back when gl_NormalMatrix still existed.