# Normal Mapping a Cubemap

I posted a question on Stackoverflow and it was suggested that it would be better asked here.

I have a sphere. It was created via the icosphere method outlined here: http://blog.andreaskahler.com/2009/06/creating-icosphere-mesh-in-code.html

I've been texturing this sphere using cubemaps.

My sphere after generation is a unit-sphere (I think that's the right term). So that it ranges between -1 to +1 on all three axis.

This is awesome because I don't need to generate any UV coords, the position of each vertex (before any transformations) IS the coordinate to sample from my cubemap.

Now I'm trying to bump map this sphere and I keep getting close, but not all the way. I believe I have two separate problems.

All of the solutions I've found to generate these require using the vertex UV texture lookup coords to orient the Tangent and Binormal on the Tangent Plane.

I have tried creating these via the same method outlined in the OpenGL spec that drivers use to determine a given 3D direction vertexes UV coord: https://www.opengl.org/registry/specs/ARB/texture_cube_map.txt

But this has given strange mottled looking results. (note the clear triangle edge in the lower left quad). So, I saw this method http://www.iquilezles.org/www/articles/patchedsphere/patchedsphere.htm which seems to give me good results.

The important parts of my Fragment shader look like this:

#version 400

layout (location = 0) out vec4 color;

struct Material
{
bool useMaps;
samplerCube diffuse;
samplerCube specular;
samplerCube normal;
float shininess;
vec4 color1;
vec4 color2;
};

struct PointLight
{
bool active;

vec3 position;
vec3 ambient;
vec3 diffuse;
vec3 specular;

float constant;
float linear;
};

uniform Material uMaterial;

smooth in vec3 ex_UV;
in vec3 ex_normal;
in vec3 ex_positionCameraSpace;
in vec3 ex_originalPosition;
in vec3 ex_positionWorldSpace;
in vec4 ex_positionLightSpace;

in PointLight ex_light;

/* ******************
Provides a better lookup into a cubemap
******************* */
vec3 fix_cube_lookup(vec3 v, float cube_size)
{
float M = max(max(abs(v.x), abs(v.y)), abs(v.z));
float scale = (cube_size - 1) / cube_size;
if (abs(v.x) != M)
v.x *= scale;
if (abs(v.y) != M)
v.y *= scale;
if (abs(v.z) != M)
v.z *= scale;
return v;
}

/* *********************
Calculates the color when using a point light. Uses shadow map
********************* */
vec3 CalcPointLight(PointLight light, Material mat, vec3 n, vec3 fragPos, vec3 originalPos, vec3 viewDir)
{
// the direction the light is in in the light position - fragpos
vec3 lightDir = normalize(fragPos - light.position);

// get the diffuse color
vec3 textureLookup = fix_cube_lookup(normalize(ex_originalPosition),textureSize(mat.diffuse,0).x);
vec3 diffuseMat = vec3(0.0);
if(mat.useMaps)
diffuseMat = texture(mat.diffuse,textureLookup).rgb;
else
diffuseMat = mat.color1.rgb;

// get the specular color
textureLookup = fix_cube_lookup(normalize(ex_originalPosition),textureSize(mat.specular,0).x);
vec3 specularMat = vec3(0.0);
if(mat.useMaps)
specularMat = texture(mat.specular,textureLookup).rgb;
else
specularMat = mat.color2.rgb;

// the ambient color is the amount of normal ambient light hitting the diffuse texture
vec3 ambientColor = light.ambient * diffuseMat;

float diffuseFactor = dot(n, -lightDir);
vec3 diffuseColor = vec3(0,0,0);
vec3 specularColor = vec3(0,0,0);
if(diffuseFactor > 0)
diffuseColor = light.diffuse * diffuseFactor * diffuseMat;

vec3 reflectDir = normalize(reflect(lightDir, n));
float specularFactor = pow(dot(viewDir,reflectDir), mat.shininess);
if(specularFactor > 0 && diffuseFactor > 0)
specularColor = light.specular * specularFactor * specularMat;

float lightDistance = length(fragPos - light.position);
float attenuation = light.constant + light.linear * lightDistance + light.quadratic * lightDistance * lightDistance;

return ambientColor + (diffuseColor + specularColor) / attenuation;
}

mat3 contangent_frame(vec3 n)
{
vec3 tangent = vec3(-(1.0 + ex_originalPosition.y*ex_originalPosition.y),ex_originalPosition.x*ex_originalPosition.y,ex_originalPosition.x);
vec3 binormal = vec3(ex_originalPosition.x*ex_originalPosition.y,-(1.0 + ex_originalPosition.x*ex_originalPosition.x),ex_originalPosition.y);

float invmax = inversesqrt(max(dot(tangent,tangent), dot(binormal,binormal)));
return mat3(tangent * invmax, binormal * invmax, n);
}

vec3 perturb_normal(vec3 n, vec3 SRT)
{
vec3 textureLookup = fix_cube_lookup(SRT,textureSize(uMaterial.normal,0).x);
vec3 map = texture(uMaterial.normal,textureLookup).rgb * 2.0 - 1.0;
mat3 TBN = contangent_frame(n);
return normalize(TBN*map);
}

void main(void)
{
vec3 srt = normalize(ex_originalPosition);
vec3 viewDir = normalize(-ex_positionCameraSpace);
vec3 PN = perturb_normal(normalize(ex_normal),srt);

vec3 result = CalcPointLight(ex_light,uMaterial,PN,ex_positionCameraSpace, ex_positionWorldSpace,viewDir);

color = vec4(result,1.0);
}


My problem is that my results get these edges on the -Y and +Y cubemap faces: There's two problems.

1. Obviously the discontinuity of the shadow, it appears that the direction of the normal is suddenly opposite.
2. The entire edge of the faces is visible. Again, the direction of the normals?

I took a capture of my normal cubemap with Renderdoc: And if you pull these faces apart and try to line them up, you'll see that the normals DO face away from each other. However, they DO line up for other faces.

So I can't just reverse the direction for the faces in question because while it fixes it for some edges, it breaks the others.

I also noticed other strange affects, like this: Where the sun is shining down on that specular section, so why would the some of the "mountains" facing it be in shadow? Should they all be lit?

This leads me to think that my normal map generation isn't "in-line" with my tangent generation. Here's my normalmap generation code. cubeFacesHeight is an std::array<std::deque<glm::vec4>,6> (6 faces of 4-component colours. The colours are grey-scale and indicate the height of the terrain.

float scale = 15.0;
for(int i = 0; i < 6; ++i)
{
std::deque<glm::vec4> normalMap(textureSize*textureSize);
for(int x = 0; x < textureSize; ++x)
{
for(int y = 0; y < textureSize; ++y)
{
// center point
int i11 = utils::math::get_1d_array_index_from_2d(x,y,textureSize);
float v11 = cubeFacesHeight[i][i11].r;

// to the left
int i01 = utils::math::get_1d_array_index_from_2d(std::max(x-1,0),y,textureSize);
float v01 = cubeFacesHeight[i][i01].r;

// to the right
int i21 = utils::math::get_1d_array_index_from_2d(std::min(x+1,textureSize-1),y,textureSize);
float v21 = cubeFacesHeight[i][i21].r;

// to the top
int i10 = utils::math::get_1d_array_index_from_2d(x,std::max(y-1,0),textureSize);
float v10 = cubeFacesHeight[i][i10].r;

// and now the bottom
int i12 = utils::math::get_1d_array_index_from_2d(x,std::min(y+1,textureSize-1),textureSize);
float v12 = cubeFacesHeight[i][i12].r;

glm::vec3 S = glm::vec3(1, 0, scale * v21 - scale * v01);
glm::vec3 T = glm::vec3(0, 1, scale * v12 - scale * v10);

glm::vec3 N = (glm::vec3(-S.z,-T.z,1) / std::sqrt(S.z*S.z + T.z*T.z + 1));

N.x = (N.x+1.0)/2.0;
N.y = (N.y+1.0)/2.0;
N.z = (N.z+1.0)/2.0;

normalMap[utils::math::get_1d_array_index_from_2d(x,y,textureSize)] = glm::vec4(N.x,N.y,N.z,v11);
}
}
for(int x = 0; x < textureSize; ++x)
{
for(int y = 0; y < textureSize; ++y)
{
cubeFacesHeight[i][utils::math::get_1d_array_index_from_2d(x,y,textureSize)] = normalMap[utils::math::get_1d_array_index_from_2d(x,y,textureSize)];
}
}
}


This code was sourced from Mathematics for 3D Game Programming and Computer Graphics, Third Edition.

So I think the problem is that my normal map is generating normals as though it was a plane, whereas it should be generating the normal as though it's an adjustment to the existing sphere-normal?

I'm also not sure that the way I'm using my newly generated Tangent and Binormal are correct. The contangent_frame function.

I just want to bump-map my cubemapped sphere without any graphical discontinuities. I had no idea I'd spend 2 weeks (and counting) on it...

(My original very long question with everything I've tried to date can be found here https://stackoverflow.com/questions/30437457/whats-wrong-with-my-normal-mapping-i-think-its-my-tangents but it's probably a good idea not to read it all)