# Why are normal maps in tangent space but not in normal space?

I want to implement normal mapping in my little game engine. When getting into normal mapping, I wonder why normal maps are typically in tangent space but not in normal space? That normal maps in object space would be complicated to handle is clear to me. But when in normal space, they could be easily added to the vertex normal, couldn't they? It seems that tangent space requires to construct a transformations matrix for each pixel.

-
What do you mean by "normal space"? How would you define this space? – Nathan Reed Apr 8 '13 at 22:14
By normal space I mean a space orthogonal to tangent space. The further difference is that a normal vector is fixed which a tangent can be rotated around the normal. In this illustration I made a offset in normal space is shown in red while a offset in tangent space is shown in blue. Maybe I haven't understand normal mapping and there is actually no distinction between normal space and tangent space? – danijar Apr 9 '13 at 8:38

A space is not just an axis though. Tangent space has 3 axes: the U tangent, the V tangent, and the normal. In your illustration you add an offset to the normal. But how do you know which direction to offset it? That's what tangent space defines. If the normal is Z, the tangent space gives you the orthogonal X and Y directions in which your offset vector lives.

Now, you could define the X and Y directions for your space in any way you like. But there's a good reason to define them using the UV mapping, which is that the normal map then makes sense visually: its red channel represents horizontal offsets - horizontal relative to the texture - and its green channel represents vertical offsets relative to the texture. It's this property of being relative to the texture that makes it possible to create a normal map matching the rest of your texture set (diffuse, specular, etc.) and apply it to surfaces of different orientations.

(BTW, the offset-vector formulation of a normal map is called a partial derivative normal map. It's handy because it offers better results for blending multiple normal maps together).

-
Thanks a lot. I had a wrong understanding of normal mapping. Also thanks for the helpful links. – danijar Apr 9 '13 at 19:43

To try and make reasonalby comple mathematical theory sort and concise: it is because the Normal is actually a co-vector rather than a vector. The difference between a covector and a vector is that a co-vector defines an opposite 'handedness' on sign reversal, while a vector defines an opposite 'direction' on sign reversal. For instance, reversing the normal to a plane defines the same plane, but with CW and CCW reversed.

For a more rigorous distinction you will have to get into the theory in fair depth.

-