2
\$\begingroup\$

I'm currently trying to implement Parallax Occlusion Mapping, based off a post on sunandblackcat.com

With my current implementation, I have the following:

i.stack.imgur.com/H0c2s.png

Notice how the parallax effect is skewed away from the camera? I can't understand what's causing this to happen, so I'm posting here in the hope that someone can help me :)

So first up, I guess I should show what my vertex data looks like:

Vertex 0

position xyz: (1, 0, -1), normal xyz: (0, 1, 0), tangent xyzw: (1, 0, 0, 1)

Vertex 1

position xyz: (-1, 0, -1), normal xyz: (0, 1, 0), tangent xyzw: (1, 0, 0, 1)

Vertex 2

position xyz: (1, 0, 1), normal xyz: (0, 1, 0), tangent xyzw: (1, 0, 0, 1)

Vertex 3

position xyz: (-1, 0, 1), normal xyz: (0, 1, 0), tangent xyzw: (1, 0, 0, 1)

This vertex data is based off the following scene, as it appears in Blender:

i.stack.imgur.com/Y3hE9.png

Note that Blender is Z-up, but my vertex data has been converted from Z-up to Y-up, hence my normals point "up" on the Y axis

Given that vertex data, I have the following vertex shader:

#version 330

uniform mat4 projectionMatrix;
uniform mat4 viewMatrix;
uniform mat4 modelMatrix;
uniform vec3 eyePos;

in vec3 in_Position;
in vec2 in_TextureCoord;
in vec3 in_Normal;
in vec4 in_Tangent;
in float in_ModelOffset;
in float in_TexOffset;

out vec2 pass_TextureCoord;
out float pass_TexOffset;
out vec3 pass_toLightInTangentSpace;
out vec3 pass_toCameraInTangentSpace;

void main(void) {
  mat3 normalMatrix = transpose(inverse(mat3(modelMatrix)));  

  pass_TexOffset = in_TexOffset;
  pass_TextureCoord = in_TextureCoord;

  // transform to world space
  vec4 worldPosition = modelMatrix * vec4(in_Position, 1);
  vec3 worldNormal = normalize(normalMatrix * in_Normal);
  vec3 worldTangent = normalize(normalMatrix * in_Tangent.xyz);

  // calculate vectors to the camera and to the light, hardcoded for now
  vec3 worldDirectionToLight = normalize(vec3(0,10,0) - worldPosition.xyz);
  vec3 worldDirectionToCamera = normalize(eyePos - worldPosition.xyz);

  // calculate bitangent from normal and tangent
  vec3 worldBitangnent = cross(worldNormal, worldTangent) * in_Tangent.w;

  // transform direction to the light to tangent space
  pass_toLightInTangentSpace = vec3(
         dot(worldDirectionToLight, worldTangent),
         dot(worldDirectionToLight, worldBitangnent),
         dot(worldDirectionToLight, worldNormal)
      );

  // transform direction to the camera to tangent space
  pass_toCameraInTangentSpace= vec3(
         dot(worldDirectionToCamera, worldTangent),
         dot(worldDirectionToCamera, worldBitangnent),
         dot(worldDirectionToCamera, worldNormal)
      );


  // calculate screen space position of the vertex
  gl_Position = projectionMatrix * viewMatrix * worldPosition;

}

And the following fragment shader:

#version 330

const int NUM_TEXTURES = 2;

uniform sampler2DArray diffuseTexture;

uniform vec3 ambientColor;

uniform float specularIntensity;
uniform float specularPower;
uniform float renderNormal;
uniform float height_scale;

in vec2 pass_TextureCoord;
in float pass_TexOffset;
in vec3 pass_toLightInTangentSpace;
in vec3 pass_toCameraInTangentSpace;

out vec4 out_Color;

const float parallaxScale = 0.1;

//////////////////////////////////////////////////////
// Implements Parallax Mapping technique
// Returns modified texture coordinates, and last used depth
vec2 parallaxMapping(in vec3 V, in vec2 T, out float parallaxHeight)
{
   // determine optimal number of layers
   const float minLayers = 10;
   const float maxLayers = 15;
   float numLayers = mix(maxLayers, minLayers, abs(dot(vec3(0, 0, 1), V)));

   // height of each layer
   float layerHeight = 1.0 / numLayers;
   // current depth of the layer
   float curLayerHeight = 0;
   // shift of texture coordinates for each layer
   vec2 dtex = parallaxScale * V.xy / V.z / numLayers;

   // current texture coordinates
   vec2 currentTextureCoords = T;

   // depth from heightmap
   float heightFromTexture = texture(diffuseTexture, vec3(currentTextureCoords, pass_TexOffset+1)).a;

   // while point is above the surface
   while(heightFromTexture > curLayerHeight)
   {
      // to the next layer
      curLayerHeight += layerHeight;
      // shift of texture coordinates
      currentTextureCoords -= dtex;
      // new depth from heightmap
      heightFromTexture = texture(diffuseTexture, vec3(currentTextureCoords, pass_TexOffset+1)).a;
   }

   ///////////////////////////////////////////////////////////

   // previous texture coordinates
   vec2 prevTCoords = currentTextureCoords + dtex;

   // heights for linear interpolation
   float nextH = heightFromTexture - curLayerHeight;

   float prevH = texture(diffuseTexture, vec3(prevTCoords, pass_TexOffset+1)).a
                           - curLayerHeight + layerHeight;

   // proportions for linear interpolation
   float weight = nextH / (nextH - prevH);

   // interpolation of texture coordinates
   vec2 finalTexCoords = prevTCoords * weight + currentTextureCoords * (1.0-weight);

   // interpolation of depth values
   parallaxHeight = curLayerHeight + prevH * weight + nextH * (1.0 - weight);

   // return result
   return finalTexCoords;
}

//////////////////////////////////////////////////////
// Implements self-shadowing technique - hard or soft shadows
// Returns shadow factor
float parallaxSoftShadowMultiplier(in vec3 L, in vec2 initialTexCoord,
                                       in float initialHeight)
{
   float shadowMultiplier = 1;

   const float minLayers = 15;
   const float maxLayers = 30;

   // calculate lighting only for surface oriented to the light source
   if(dot(vec3(0, 0, 1), L) > 0)
   {
      // calculate initial parameters
      float numSamplesUnderSurface = 0;
      shadowMultiplier = 0;
      float numLayers = mix(maxLayers, minLayers, abs(dot(vec3(0, 0, 1), L)));
      float layerHeight = initialHeight / numLayers;
      vec2 texStep = parallaxScale * L.xy / L.z / numLayers;

      // current parameters
      float currentLayerHeight = initialHeight - layerHeight;
      vec2 currentTextureCoords = initialTexCoord + texStep;
      float heightFromTexture = texture(diffuseTexture, vec3(currentTextureCoords, pass_TexOffset+1)).a;
      int stepIndex = 1;

      // while point is below depth 0.0 )
      while(currentLayerHeight > 0)
      {
         // if point is under the surface
         if(heightFromTexture < currentLayerHeight)
         {
            // calculate partial shadowing factor
            numSamplesUnderSurface += 1;
            float newShadowMultiplier = (currentLayerHeight - heightFromTexture) *
                                             (1.0 - stepIndex / numLayers);
            shadowMultiplier = max(shadowMultiplier, newShadowMultiplier);
         }

         // offset to the next layer
         stepIndex += 1;
         currentLayerHeight -= layerHeight;
         currentTextureCoords += texStep;
         heightFromTexture = texture(diffuseTexture, vec3(currentTextureCoords, pass_TexOffset+1)).a;
      }

      // Shadowing factor should be 1 if there were no points under the surface
      if(numSamplesUnderSurface < 1)
      {
         shadowMultiplier = 1;
      }
      else
      {
         shadowMultiplier = 1.0 - shadowMultiplier;
      }
   }
   return shadowMultiplier;
}

//////////////////////////////////////////////////////
// Calculates lighting by Blinn-Phong model and Normal Mapping
// Returns color of the fragment
vec4 normalMappingLighting(in vec2 T, in vec3 L, in vec3 V, float shadowMultiplier)
{
   // restore normal from normal map
   vec3 N = normalize(texture(diffuseTexture, vec3(T, pass_TexOffset+1)).xyz * 2 - 1);
   vec3 D = texture(diffuseTexture, vec3(T, pass_TexOffset)).rgb;

   // ambient lighting
   float iamb = 0.2;
   // diffuse lighting
   float idiff = clamp(dot(N, L), 0, 1);
   // specular lighting
   float ispec = 0;
   if(dot(N, L) > 0.2)
   {
      vec3 R = reflect(-L, N);
      ispec = pow(dot(R, V), 32) / 1.5;
   }

   vec4 resColor;
   resColor.rgb = D * (vec3(0.1, 0.1, 0.1) + (idiff + ispec) * pow(shadowMultiplier, 4));
   resColor.a = 1;

   return resColor;
}

/////////////////////////////////////////////
// Entry point for Parallax Mapping shader
void main(void)
{
   // normalize vectors after vertex shader
   vec3 V = normalize(pass_toCameraInTangentSpace);
   vec3 L = normalize(pass_toLightInTangentSpace);

   // get new texture coordinates from Parallax Mapping
   float parallaxHeight;
   vec2 T = parallaxMapping(V, pass_TextureCoord, parallaxHeight);

   // get self-shadowing factor for elements of parallax
   float shadowMultiplier = parallaxSoftShadowMultiplier(L, T, parallaxHeight - 0.05);

   // calculate lighting
   out_Color = normalMappingLighting(T, L, V, shadowMultiplier);
} 

I think the majority of this lines up with the sunandblackcat implementation, although I load in my textures as a texture array - the first texture being an RGBA diffuse map, and the second texture being a normal map (RGB) and height map (A)

From what I've posted, can anyone tell me where I'm going wrong, and why my parallax effect is skewed away from the camera?

Thanks

* Update *

So I decided to cut out the middle man (my exporter), and manually tweaked the tangents to see how far it got me. So after some brute forcing, for a simple plane, it seems that a tangent xyzw of (-1, 0, 0, -1) gave me something that looked like parallax occlusion mapping:

Acceptable

But when I move in quite close to the mesh, the effect skews again:

Not so nice

Is this an expected side effect of POM, or does this indicate an issue with how the effect is being calculated?

As for the updated tangents - I'm not sure if I need to update my exporter to negate the xyzw components, or if only x and w need to be negated - but I don't really understand why I need to modify the tangents at all? Given that even with the updated tangents, there are still some strange effects happening up close to the surface, I can't help but feel that there's something going wrong in my shaders...

\$\endgroup\$
  • \$\begingroup\$ It looks like the shader and your model might disagree about which way the tangent basis should point. Effectively, according to the math in your shader, the camera is actually somewhere to the left (off the "West" corner of your mesh) while your mesh / vertex transform pipeline thinks the camera is off the "South" corner. You might just need to rotate your tangent basis 90 degrees to bring the two conventions into agreement. \$\endgroup\$ – DMGregory Jan 24 '17 at 3:39
  • \$\begingroup\$ @DMGregory so if I apply the Z-up to Y-up conversion to my tangent, the same way I do for my position and normal, I get the following: [ imgur.com/a/8YaUd ] Although looking at my vertex data, I can see my tangent is now (-1, 0, 0, 1), which is a rotation of 180, not 90 degrees like you said. How exactly should I rotate 90 degrees the "correct" way? Should I just swap X and Z? Or will that break down somewhere? Thanks for your reply! \$\endgroup\$ – totbl Jan 24 '17 at 9:34
  • \$\begingroup\$ In blender when you export there's usually a way to choose the up direction \$\endgroup\$ – Bálint Jan 24 '17 at 11:56
  • \$\begingroup\$ @Bálint yes I convert from z up to y up in my exporter, so I think my vertex positions are correct, and it makes sense to me that my normals are pointing up along Y, I'm just not sure how I should rotate my tangents, or if it even matters. I suppose beyond that, I'm not really sure why I have to rotate my tangents - surely Y would be up in the sunandblackcat implementation? \$\endgroup\$ – totbl Jan 24 '17 at 13:06
  • \$\begingroup\$ No, blender's export has an option for the vertical axis, you don't need to do much about that \$\endgroup\$ – Bálint Jan 24 '17 at 13:16
3
\$\begingroup\$

normalMapping should use TBN, Parallax Mapping should use transpose(TBN), and viewDirection should mul transpose(TBN).

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.