SSAO, depth buffer linearization (?)

Question

I'm trying to implement SSAO in my application but it doesn't look as it should. I think that the problem is with depth buffer linearization but I tried almost all methods which I found on the Web and each gave me different and wrong results.

---

Depth buffer

My application implements deferred rendering. My framebuffer which represents G-Buffer contains GL_DEPTH_STENCIL_ATTACHMENT with texture attached to it.

It's base and internal format is GL_DEPTH_STENCIL. Used data type is GL_UNSIGNED_INT_24_8.

Parameters which I set to this texture are:

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_R, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_NONE);

Here is the content of this buffer drawed on screen:

---

SSAO

Here is my SSAO shader:

#version 330

uniform sampler2D nrm;
uniform sampler2D depth;
uniform sampler2D noise;

uniform mat4 ProjectionMatrix; // current projection matrix, for linearized depth

//  ssao uniforms:
const int MAX_KERNEL_SIZE = 64;
uniform int kernelSize = 16; // 20
uniform float power = 2.0;
uniform vec3 kernel[MAX_KERNEL_SIZE];  // wasnt here

uniform vec2 texelSize;
out vec4 FragColor;

vec3 decode_normal(vec2 enc){
    vec2 fenc = enc*4-2;
    float f = dot(fenc,fenc);
    float g = sqrt(1-f/4);
    vec3 n;
    n.xy = fenc*g;
    n.z = 1-f/2;
    return n;
}

vec2 CalcTexCoord(){
    return gl_FragCoord.xy*texelSize;
}

vec3 posFromDepth(in vec2 Tex, in float d){ // get eye coordinate from depth
    vec3 pos = vec3(Tex, d); 

    // transform by the projection inverse 
    vec4 clip = inverse(ProjectionMatrix)*vec4(pos*2-1,1);

    // divide by w to get the position. 
    return clip.xyz/clip.w;
}

float ssao(){
    vec2 Tex = CalcTexCoord();
    float radius = 3.0;

    //  get noise texture coords:
    vec2 noiseTexCoords = vec2(textureSize(depth, 0)) / vec2(textureSize(noise, 0));
    noiseTexCoords *= Tex;

    float originDepth = texture(depth, Tex).z;
    vec3 originPos = posFromDepth(Tex, originDepth);

    float occlusion = 0.0;

    float sampleDepth = 0.0;
    for (int i = 0 ; i < kernelSize ; i++) {
        //  get sample position:
        vec3 samplePos = originPos + kernel[i];

        vec4 offset = ProjectionMatrix * vec4(samplePos, 1.0);
        offset.xy /= offset.w;
        offset.xy = offset.xy * 0.5 + vec2(0.5);

        sampleDepth = texture(depth, offset.xy).z;
        sampleDepth = posFromDepth(offset.xy, sampleDepth).z;

        float rangeCheck = smoothstep(0.0, 1.0, radius / abs(originPos.z - sampleDepth));
        occlusion += step(sampleDepth,samplePos.z)*rangeCheck;
    }

    occlusion = (occlusion/float(kernelSize));

    return pow(occlusion, power);
}

void main() {
    FragColor = vec4(ssao());
}

As you can see I reconstruct position from depth. In other shaders this method works ok but here result looks like this:

---

Other methods which I tried

I tried also other methods which was presented in many tutorials but it gaves me even worse effects (for example everything was black). Here are some of methods which I used:

float CalcViewZ(vec2 Coords)
{
    float Depth = texture(depth, Coords).x;
    return ProjectionMatrix[3][2] / (2 * Depth -1 - ProjectionMatrix[2][2]);
}

float ViewZ = CalcViewZ(Tex);

float ViewX = viewRay.x * ViewZ;
float ViewY = viewRay.y * ViewZ;

...

vec3 originPos = vec3(ViewX, ViewY, ViewZ);

sampleDepth = CalcViewZ(offset.xy);

and

float linearizeDepth(in float z, in mat4 projMatrix){
    float n = 0.1;
    float f = 90.0;

    //return f * n / (z * (f - n) - f);
    //return (2.0 * n) / (f + n - z * (f - n)); 
    //return n * (z + 1.0) / (f + n - z * (f - n));
    return (2* n * f) / (f + n - z * (f - n));
}

float linearizeDepth(in float depth, in mat4 projMatrix) {
    return projMatrix[3][2] / (depth - projMatrix[2][2]);
}

...

float originDepth = texture(depth, Tex).z;
originDepth = linearizeDepth(originDepth, ProjectionMatrix);
vec3 originPos = viewRay * originDepth;

float sampleDepth = texture(depth, offset.xy).z;
sampleDepth = linearizeDepth(sampleDepth, ProjectionMatrix);

where viewRay is created in vertex shader:

viewRay = vec2(
    Vertex.x * uTanHalfFov * uAspectRatio,
    Vertex.y * uTanHalfFov
);

I have currently no ideas what is wrong with this. It's a bit urgent to me and I'll appreciate any help.

Edit

I used linearization given by @Andon M. Coleman but sadly it doesn't help. Here are results:

float linearizeDepth(in float depth) {
    float n = 0.1;
    float f = 90.0f;
    return n / (f - depth * (f - n)) * f;
}

float originDepth = texture(depth, Tex).z;
originDepth = linearizeDepth(originDepth);
vec3 originPos = viewRay * originDepth;

sampleDepth = texture(depth, offset.xy).z;
sampleDepth = linearizeDepth(sampleDepth);

Here is how I'm creating Projection matrix (maybe it will help):

_projection3D.x.x=  (2 * from) / (right - left);
_projection3D.z.x= (right + left) / (right - left);
_projection3D.y.y=  (2 * from) / (top - bottom);
_projection3D.z.y= (top + bottom) / (top - bottom);
_projection3D.z.z=-(to + from) / (to - from);
_projection3D.w.z=-(2 * to * from) / (to - from);
_projection3D.z.w= -1;

Answer 1

Solution

I used good algorithm for depth linearization but I didn't convert depth to NDC first:

const float near = 0.1; // projection matrix's near plane
const float far = 90.0; // projection matrix's far plane
float LinearizeDepth(float depth)
{
    float z = depth * 2.0 - 1.0; // back to NDC 
    return (2.0 * near * far) / (far + near - z * (far - near));    
}

I also had another problem. Generated AO texture has strange visible strips. The solution to this problem was to set GL_TEXTURE_MIN_FILTER and GL_TEXTURE_MAG_FILTER to GL_LINEAR instead of GL_NEAREST.

Here is final effect with blur:

Because currently I store AO only in red channel I removed other channels from image. The one thing which I think should be better is too blurry background but I'm not sure how to fix it now (blur dependent on depth?).

I'd like also to put here link to fantastic page about OpenGL which helped mi a lot with my SSAO problems: Learn OpenGL -> SSAO