# For normal mapping, why can we not simply add the tangent normal to the surface normal?

I am looking at implementing bump mapping (which in all implementations I have seen is really normal mapping), and so far all I have read says that to do this, we create a matrix to convert from world-space to tangent-space, in order to transform the lights and eye direction vectors into tangent space, so that the vectors from the normal map may be used directly in place of those passed through from the vertex shader.

What I do not understand though, is why we cannot just use the normalised sum of the sampled-normal vector, and the surface-normal? (assuming we already transform and pass through the surface normal for the existing lighting functions)

Take the diagram below; the normal is simply the deviation from the 'reference normal' for any given coordinate system, correct? And transforming the surface normal of a mapped surface from world space to tangent space makes it equivalent to the tangent space 'reference normal', no?
If so, why do we transform all lighting vectors into tangent space, instead of simply transforming the sampled tangent once in the pixel shader?

First of all, re: "why we cannot just use the normalised sum of the sampled-normal vector, and the surface-normal?" If they're in the same space already, summing these two just has the effect of halving the strength of the normal map - it effectively blends 50% between the normal map and the non-normal-mapped geometric normals. If they're not in the same space, then the sum makes no sense and would just end up doing something odd, like biasing all your normals upward and screwing up your shading.

As for the space in which to do shading, it can actually be done in various ways - in tangent space, or in world space*, or in another space such as view space.

One possible reason to do it in tangent space is that all the other relevant vectors - camera, light directions, etc. - can be transformed into tangent space in the vertex shader, saving work (except for some normalizes) in the pixel shader. Shading in another space means you have to apply a transform per-pixel to the sampled normal.

On the other hand, modern lighting models often include not just point-lights but lighting environments in the form of cubemaps, spherical harmonics, and suchlike, which are going to be defined relative to world space. So if you did shading in tangent space you'd have to throw in a bunch more per-pixel transforms to get the vectors for these things back into world space.

On the whole I generally prefer shading in world space at present.

*Note that "world space" for shaders usually should have its origin shifted to the camera to assist with numerical precision issues, so it's really camera-centered world space, not true world space. Rotationally, it's still aligned with the world axes.

Because adding two vectors together and normalizing the result will give you a vector that is halfway between the two of them. I don't think that corresponds to what you were thinking it would do.

Take for instance the following picture from an unrelated subject (Blinn-Phong shading model) and pay attention to the H vector:

The H vector is the result of adding V and L and normalizing the result, which is pretty much what you were trying to do.

So, back to your problem, it should be clear from that diagram that simply adding the tangent space sampled normal and the world space surface normal together and normalizing the result won't be the same as applying the correct displacement to the surface normal as you expected.

It will just return you some new vector that is halfway between both of them, and since both vectors aren't in the same space, the result wouldn't even make any sense.

So it's important to do the conversion and work in the same space, either in tangent space or world space. Nathan gave the pros and cons of working in each of these spaces in his answer.

• Thanks for your answer, I think I will now go reread that introduction to vector maths after this mistake! – sebf Mar 20 '12 at 10:19

Take the diagram below; the normal is simply the deviation from the 'reference normal' for any given coordinate system, correct?

No, it is not.

The simplest way to understand this is to take the simplest possible case of bump mapping. Your geometry is a flat quad. And you're going to apply a normal map to this quad. Now, let's say that all of the normals in this texture have a 45-degree angle pointing up. That may not be meaningful, but let's just say that this is what our normals are.

OK, so... what direction is up?

There are many ways to map a square texture onto a square quad. Maybe the texture coordinates point the top of the texture in the +Y direction of the quad. Maybe they point it in the +X direction of the quad. Maybe the -Y direction.

All of these answers must produce a different normal when accessed from the texture. After all, if you rotate the texture mapping by 90-degrees, the normal you fetch from the texture had better rotate with it, right? The orientation of the texture relative to the model is therefore important.

That's what the tangent-space basis matrix does. It provides a transformation from model space (not world-space) to tangent-space. It specifies what the orientation of the normals are.

• Thank you, this makes a lot of sense as to why a 'complete' transform between the coordinate systems is needed. – sebf Mar 20 '12 at 10:19