# How does normal mapping really work?

I'm trying to grasp the concept of normal mapping, but I'm confused by a few things. In short, I'm not sure whether a normal map is viewpoint dependent or not (i.e. whether you'll get a different normal map of the same object when you rotate around it). Secondly, I don't get why the blueish color is the predominant color in normal maps.

How I think about normals, and their relation to RGB colors, is as follows. The unit sphere represents any unit normal possible — in other words, the X, Y and Z components of a unit normal vector range from -1 to 1. The components of an RGB color all range from 0 to 255. Therefore, it makes sense to map -1 (normal component) to 0 (color component), 0 to 127 or 128, and 1 to 255. Any value in between is just linearly interpolated.

Applying this mapping to the normals of an arbitrary 3D object results in a very colorful picture, not at all predominantly blue. For example, when taking a cube, all six faces would have a different, but uniform, color. For instance, the face with the normal (1,0,0) would be (255,128,128), the face with the normal (0,0,-1) would be (128,128,0) and so on.

However, for some reason the normal maps of a cube I found are completely blueish, i.e. (128,128,255). But clearly, the normals are not all in the positive z-direction, i.e. (0,0,1). How does this work?

Ok, so the approach described above seems to be referred to as the object space normal map or the world space normal map. The other one is called the tangent space normal map. I understand how such a tangent space normal map can be used to modify the normals of a geometry, but I'm still not completely sure how it is actually calculated (see my comment at Nicol Bolas' answer).

[Edit 2]

I should probably mention that I'm working with piecewise parametric surfaces. These surfaces consist of a set of surface patches, where each patch is associated with its own parametric space (u,v) = [0,1] x [0,1]. At any point on the surface, the normal can be calculated exactly. Apparently, the vectors T (tangent) and B (bi-tangent) — required to span the tangent space — are not simply the partial derivatives of the surface patch in the direction of u and v...

• Have you ever tried to implement phong or at least diffuse shading? Do you know basic formulas like dot(a,b) = cos(angle(a,b)) for unit vectors a,b ? Few hours of implementing would save you years of struggling. – Ivan Kuckir Jun 8 '13 at 16:10
• @Ailurus: see my comment/answer on how to approach the same problem for terrain objects/height fields. It does get a tad more complicated for other objects, but the concepts shouldn't differ. – teodron Jun 9 '13 at 9:27
• The particular case of terrain normal mappping: gamedev.stackexchange.com/questions/43894/… – teodron Jun 10 '13 at 19:21

A texture mapping is the mapping between points on the 3D surface and their corresponding points on a texture image. If you have a 1:1 texture mapping, then every point on the 3D surface maps to a specific and unique point in the texture image (though the reverse would not need to be true. Some locations in the texture would not necessarily map to locations on the surface).

With such a mapping, you could go through the 3D surface and store each distinct normal in the corresponding location in the texture.

OK fine, let's do that. We'll go through a 3D surface and generate object-space normals the mapped locations, and then stick them in the texture. So when we want to render, we simply fetch the object-space normal from the texture and we're done. Right?

Well yes, that would work. But it also means that the texture's normals can only ever be used with that particular object. And it also means that the texture's normals can only be used with that object and with that specific texture mapping. So if you wanted to rotate the texture mapping in some way, or alter it with some UV transform, you're out of luck.

So generally, what people use are normal maps where the normals are in "tangent-space". Tangent-space is the space relative to the mapped point on the 3D surface, where the unmodified normal is in the +Z direction, and the X and Y axes point along the U and V axes relative to the surface.

Tangent-space essentially regularizes the normals. In tangent-space, the normal (0, 0, 1) always means "unmodified"; it's the normal you get from interpolating the vertex normal. This leads to a number of useful things you can do, one of the most important of which is to store them in less data.

Since the Z will always be positive, you can therefore compute it in your shader from the X and Y components. Since you only need 2 values, so you could use (in OpenGL image format nomenclature) GL_RG8, a 2-byte-per-pixel format rather than GL_RGBA8, a 4-byte-per-pixel (GL_RGB8 will still be 4-bytes-per-pixel, since GPUs will pad each pixel out to 4 bytes). Even better, you can compress those two values, leading to a 1-byte-per-pixel format. So you've reduced your texture's size to 75% of the object-space normal map.

Before you can talk about any kind of normal map, you need to know first what it stores. Is it an object-space normal map, a tangent-space normal map, or something else?

• Alright, so the first type of map you describe is an object-space normal map, right? This is the more colourful one, because it stores the real X, Y and Z components of the normals. The second type, the tangent-space normal map, seems to store something like the perturbations to the normals rather than the normals themselves. I'll read up on the concept of a tangent space and check back later. – Ailurus Jun 8 '13 at 13:57
• Ok, so the tangent space of a point on the surface is simply the space of all tangent vectors of that point. However, in that case I don't see how the normal as seen from the tangent space can be anything else than (0,0,1)? In other words, I would always expect the normal map to be uniformly blue (128,128,255). Yet, the example shown on the wikipedia page (en.wikipedia.org/wiki/Normal_mapping#How_it_works) contains other colors. I do understand how this map can be used to change the normals, but not how it is actually calculated. – Ailurus Jun 8 '13 at 23:11
• @Ailurus: "Ok, so the tangent space of a point on the surface is simply the space of all tangent vectors of that point." No, it isn't. It is the space defined by the (unmodified) normal and the direction of the texture coordinates at that point on the surface. It regularized the normal stored in the texture as being relative to the texture mapping and the current normal before modifications. – Nicol Bolas Jun 8 '13 at 23:18
• Probably the most difficult part about a normal map is the mapping itself (including how to compute the tangent vectors). Isn't the computation of the tangent vectors intrinsically linked to the UV mapping/wrapping of the diffuse texture AND to the geometry of the object altogether? Put it another way: you have two different normal maps for an object, but only one tangent (and normal) field computed. You may find out you can't use the tangent field consistently with two different normal textures (although geometrically you could find suitable uv-mappings, which isn't a trivial task). – teodron Jun 9 '13 at 9:24
• @NicolBolas Ah, I got confused because some sources claim that the vectors T, B and N form an orthonormal basis, whereas others mention that this isn't necessarily true. – Ailurus Jun 9 '13 at 13:41

Normal maps are mapped using the so-called tangent space, which is essentially a local space based on the model's texture-space. This should answer both of your questions.

It's not viewpoint dependant because this space has nothing to do with the camera. In the normal map, Z is the up direction. If you look at the normals of a model, most of the normal vectors will be pointing directly out from the mesh. The mesh's surface is the texture space I was talking about, so in that local coordinate system, up is the "outward" direction.

• Thanks for clarifying this! However, I'm not using a polygonal mesh as surface, but a piecewise smooth parametric surface. Therefore, all normals would be saved as pointing in the (0,0,1) direction, right? – Ailurus Jun 9 '13 at 14:00

Look at the right hand B/W drawing under datasets - this is (or at least used to be) known as a hedgehog drawing, a rendering of a surface with each of it's normals drawn

So, to understand a traditional normal map, think of an upset hedgehog with all their spines sticking out - each of those spines is normal to the surface of the hedgehog underneath it -

With respect to your sphere question, if you were living purely in parametric space, as with ray tracers, then the infinite set of normals to the sphere would simply create a larger sphere - in tessellated space, i.e. the world the computer forces on us if we want real time, then you have a spiky approximation of a sphere.

Now, this example focused on an OBJECT normal map - it's defining normals with respect to the object, and this is invariant under any rotation, translation, or scaling - of either the object or the camera or anything else - as mentioned earlier, this is only one kind of normal map but it's the most common

I think you might have some misconceptions on what normal maps are. Basically, it's a way of simulating the look of something bumpy when in fact the geometry is completely flat.

The colours of a normal map are interpreted by a tech shader and processed based on light intensity and direction, as well as your camera view. This means that you could have a brick floor for example, which is completely flat, with a flat texture, but because it has a normal map with the same brick shape, when you look around it, light will appear to be bouncing off the side of the bricks making it look more 3D than it is.

This is of course just an illusion, but it's much cheaper than having complex geometry. And no, the colours of the normal map don't change. They really just represent values to compare against in the shader. I'm sure someone here will be able to fill you in in much more detail.

• I know, but thanks anyway for your answer :) – Ailurus Jun 9 '13 at 14:02