Correctly transforming normals for G-Buffer in Deferred Rendering

Question

I have a deferred rendering system: first there is the G buffer generation, then there is the lighting or shading calculation on that gbuffer data.

However there seems to be an issue with various rendering algorithms and I can’t determine why that is. I think it’s because my normal’s Gbuffer texture or other G buffer texture does not have the proper format or space in terms of world space, tangent space etc.

The algorithms that rely on normal information such as normal oriented ambient occlusion and parallax occlusion mapping don’t work they just look wrong And not what they’re supposed to be by a long shot.

In the G buffer generation stage, my vertex shader simply passes along world space vertex positions and world space normals to the Gbuffer's fragment shader. I do this by multiplying the world matrix or model matrix by the normal and by the position. In the fragment shader I then write these out as is to the G buffer.

Is this correct? Should I be passing along the Raw position & normals to the fragment shader and then multiplying the matrices in the fragment shader instead?

Edit 1:

Ok, So using the "normalMatrix" didn't change anything. Here is a screen shot of my ambient-occlusion pass demonstrating the issue. There is a screen space split that basically inverts the colors. and some other oddities.

The SSAO Shader is As Follows.

void main()
{

vec3 WorldPosition = texture(GPositionsTexture, In_TexCoords).xyz;
vec3 WorldNormal = normalize(UnScaleNormal( texture( GNormalsTexture, In_TexCoords ).xyz ));

vec3 Random = texture(KernalNoiseTexture, In_TexCoords * KernelNoiseScale).xyz;
vec3 Tangent = normalize(Random - WorldNormal * dot(Random, WorldNormal));
vec3 BiTangent = cross(WorldNormal, Tangent);
mat3 TBN = mat3(Tangent, BiTangent, WorldNormal);

float Occlusion = 0.0;
for (int i = 0; i < KERNEL_SIZE; ++i)
{
    // get sample position:
    vec3 Sample = TBN * Kernel[i];
    Sample = Sample * Radius + WorldPosition;

    // project sample position:
    vec4 Offset = vec4(Sample, 1.0);
    Offset = CamData.Projection * CamData.View * Offset;
    Offset.xy /= Offset.w;
    Offset.xy = Offset.xy * 0.5 + 0.5;

    // get sample depth:
    float SampleDepth = texture(GPositionsTexture, Offset.xy).z;

    // range check & accumulate:
    float RangeCheck= abs(WorldPosition.z - SampleDepth) < Radius ? 1.0 : 0.0;
    Occlusion += (SampleDepth <= Sample.z ? 1.0 : 0.0) * RangeCheck;
}

Occlusion = 1 - (Occlusion / KERNEL_SIZE);

Out_Diffuse = pow(Occlusion, Power);
}

Answer 1

In the case of a deferred system, your pipeline is as follows:

1. Compute projection matrix
2. Compute view matrix
3. Compute world to screen ( mat4 pm = projection * view )

4. Clear Geometry/Depth buffers.

5. Then, for each scene object, compute their world space transforms, and normal matrices. Tangent space (TBN) matrices can be computed in the first pass shader. The normal matrix is the inverse transpose of the world space transform(not object space to view space, as you would in a simpler forward rendering pipeline).

6. Activate your Geometry buffer, and draw as normal.

This next part is very important:

7. Compute the screen space position as normal (vec4 s = pm * world * vertex), in the vertex shader, along with your per vertex TBN (tangent space) matrix, this is obviously used for normal/parallax mapping. Your Bi/Co tangents should be computed when you load your meshes, for a faster, more efficient pipeline.

The data that gets written to each buffer is in world space, at screen space position for each buffer target.

Buffer0 is world space positions, buffer1 is world space normals. The rest are colour, and emissive properties, depending on your lighting model (let's take a simple example). Buffer2 rgb values are diffuse colour, and the alpha value is reflectity (or clamped specular value). Buffer3 is glow/emissive. Any additional buffer textures are for whatever else you may need.

8. Then you bind the GBuffer textures for sampling, and activate your swap buffer(composed of two buffer textures) and shaders.

Lighting calculations are performed in world space, by sampling all the information they need from the buffers.

9. Then, if you want to perform, additional post processing(HDR/AA etc), you swap and clear your swap buffers for each post process.

10. When you are done, finally, render a full screen quad, with the last written buffer as it's texture, directly to the back buffer.

This is the simple version, bereft of specific technical details, but I think that should answer any general questions you may have, and give you what you need to google anything else that pops up.

Answer 2

So, Using the "NormalMatrix" actually did solve many of my issues with other algorithms that use them such as Parallax Occlusion Mapping.

However, there still remains an issue with my Ambient Occlusion algorithm so i'm going to create a different question specific to that. Thanks :)