Rotating a vector by another vector in shader

Question

I have a terrain surface with a normal for each point on the terrain.

I have a second detail normal map to be applied to the terrain.

• These normals are in 3-space.

• The Y value of both normals is always positive.

• The X,Z values of both normals can be positive/negative/zero.

• The 1st normal vector (blue) that we are rotating the 2nd vector (orange) by, can be almost horizontal.

I am okay with an approximate solution if it makes it easier/faster to compute.

In the above image, you see the blue surface normal (from the 1st normal map), the orange normal map normal (from the 2nd normal map), and the green desired result normal.

The amount that the orange vector is rotated should be roughly (or if possibly exactly) equal to the angle that the blue normal vector forms with the XZ plane (where Y is up, like the DirectX coordinate system).

Here is a second scenario:

In the above image, the blue surface normal is almost horizontal, so the 2nd normal map is being applied to an almost vertical surface, thus the orange normal map vector is rotated further.

The rotation is being implemented in a HLSL shader.

How do I rotate the 1st orange normal based on the direction of the 2nd blue normal?

Edit: Maybe I need a tangent and bitangent as well as the normal?

Here's how I get the normal:

float4 ComputeNormals(VS_OUTPUT input) : COLOR
{
    float2 uv = input.TexCoord;

    // top left, left, bottom left, top, bottom, top right, right, bottom right
    float tl = abs(tex2D(HeightSampler, uv + TexelSize * float2(-1, -1)).x);
    float  l = abs(tex2D(HeightSampler, uv + TexelSize * float2(-1,  0)).x);
    float bl = abs(tex2D(HeightSampler, uv + TexelSize * float2(-1,  1)).x);
    float  t = abs(tex2D(HeightSampler, uv + TexelSize * float2( 0, -1)).x);
    float  b = abs(tex2D(HeightSampler, uv + TexelSize * float2( 0,  1)).x);
    float tr = abs(tex2D(HeightSampler, uv + TexelSize * float2( 1, -1)).x);
    float  r = abs(tex2D(HeightSampler, uv + TexelSize * float2( 1,  0)).x);
    float br = abs(tex2D(HeightSampler, uv + TexelSize * float2( 1,  1)).x);

    // Compute dx using Sobel filter.
    //           -1  0  1 
    //           -2  0  2
    //           -1  0  1
    float dX = tr + 2*r + br - tl - 2*l - bl;

    // Compute dy using Sobel filter.
    //           -1 -2 -1 
    //            0  0  0
    //            1  2  1
    float dY = bl + 2*b + br - tl - 2*t - tr;

    // Compute cross-product and renormalize
    float3 N = normalize(float3(-dX, NormalStrength, -dY));    

    // Map [-1.0 , 1.0] to [0.0 , 1.0];
    return float4(N * 0.5f + 0.5f, 1.0f);
}

How might I get the tangent and bitangent vectors then?

Is it enough to take the cross product of the normal with the Z axis unit vector to find the tangent vector? (Since the normal.Y is always positive, where Y is up, and Z is pointing into your screen).

And then take that tangent vector and cross it with the normal to obtain the bitangent?

And then take the normal, tangent and bitangent together to form a rotation matrix to rotate the orange normal map vector?

Even if that works, this seems like a lot of computation for a pixel shader. Does this work, and is there a more efficient way?

Edit:

This image may help you understand what I'm trying to do:

If you know what normal mapping is, then this should be straightforward, I think.

I'm trying to take the normal map, which contains various normals, and apply them to a surface. The surface has it's own normals. The normal map will contain many more normals than the surface, so several normal map normals are sampled across a single part of the surface that has only a single normal.

Answer 1

I think I get your question, although the title and the 2 first pictures are confusing.

The problem is, there is a normal-map to be mapped on a xz-plane (the orange vector). Now the actual plane described by the blue normal is not necessarily in the xz-plane, so one must find a rotation from the xz-plane to the actual plane, in order to rotate the orange vector to map it on the actual plane.

Now how can we do that? Bacially we must find the rotation, that rotates the xz-plane to the plane described by the blue normal.

1. Find the rotation-axis by taking the cross-product of the blue normal and the normal from the xz-plane {0, 1, 0}, the orange vector must be roatated around this axis.

   axis = N_xz cross N_blue
   

2. Find the rotation angle by taking arccosine of the dot-product of both normals

   angle = acos(N_xz dot N_blue)
   

3. Now create a rotation-matrix from those values and transform the orange normal with that matrix.

Answer 2

I believe you're over thinking your problem.

To get back to standard normal mapping, you normally have 3 vectors: the surface normal, a tangent vector perpendicular to that one, and a cotangent perpendicular to both of those. Together, these form the tangent space in which the normal map describes a direction. With this setup, you just have to multiply each component read of the normal map with its respective tangent space axis, and sum the results.

So you have a terrain, and a normal map you'd like to apply to it. You've already worked out how to take the delta height and generate a surface normal. You're now looking for the tangent and cotangent.

So what IS a tangent? Well, from the description above, it's actually the 3D direction described by one of the UV axes in your texture mapping. After all, that's what it gets used for, right? So how do you find this direction? Well, given an arbitrary mesh, it's a little tricky; but given a regular grid terrain, it's self evident: if you have a regular grid with regular UVs, then the tangent and cotangent directions are just the two axis aligned edges leaving your vertex.

So, you know what the heights of points around you are. You can pass in how far away these points are on the plane. Subtract those from the point you're on, normalize, and there you go: instant tangent space.

Answer 3

If you do have the tangent also, you can use this function to generate the bumped normal you are after. Otherwise, feel free to plagiarise in future!

//------------------------------------------------------------------------------
// Transforms a normal map sample to world space.
//------------------------------------------------------------------------------


  float3 NormalSampleToWorldSpace(float3 normalMapSample, float3 unitNormalW, float3 tangentW)
    {
        // Uncompress each component from [0,1] to [-1,1].
        float3 normalT = 2.0f * normalMapSample - 1.0f;

        // Build orthonormal basis.
        float3 N = unitNormalW;
        float3 T = normalize(tangentW - dot(tangentW, N) * N);
        float3 B = cross(N, T);

        float3x3 TBN = float3x3(T, B, N);

        // Transform from tangent space to world space.
        float3 bumpedNormalW = mul(normalT, TBN);

        return bumpedNormalW;
    }

Answer 4

You can use the LERP function, this will linearly interpolate between the two vectors. Giving you a nice 'averaged' normal.

http://msdn.microsoft.com/en-us/library/bb509618(VS.85).aspx

(Of course this is not really rotating, but given the orange and blue vector this will get you the green vector)