FAST position reconstruction from depth

Question

I'm struggling a lot with reconstructing the world space position (or alternatively the view space position) from depth (by reading the depth buffer) in a performant way (in a full-screen post-process).

So far I've been using this approach:

vec3 calculate_world_position(vec2 texture_coordinate, float depth_from_depth_buffer)
{
    vec4 clip_space_position = vec4(texture_coordinate * 2.0 - vec2(1.0), 2.0 * depth_from_depth_buffer - 1.0, 1.0);

    //vec4 position = inverse_projection_matrix * clip_space_position; // Use this for view space
    vec4 position = inverse_view_projection_matrix * clip_space_position; // Use this for world space

    return(position.xyz / position.w);
}

While this works perfectly, it's really slow. What I'm looking for is an approach like this:

vec3 world_position = world_space_eye_position + view_direction * depth;

I have found a couple of results on google but most are based on DirectX and I didn't understand them and therefore I have failed to implement them.

Answer 1

I found a solution that works well. I don't think it gets any faster than this.

What I did first is, I did the matrix-vector multiply by manually and skipped all the matrix components which are zero. I used an inverse projection matrix for this (to transform to view space). That left me with this code:

vec3 calculate_view_position(vec2 texture_coordinate, float depth_from_depth_buffer)
{
    vec3 clip_space_position = vec3(texture_coordinate, depth_from_depth_buffer) * 2.0 - vec3(1.0);

    vec4 view_position = vec4(vec2(inverse_projection_matrix[0][0], inverse_projection_matrix[1][1]) * clip_space_position.xy,
                                   inverse_projection_matrix[2][3] * clip_space_position.z + inverse_projection_matrix[3][3]);

    return(view_position.xyz / view_position.w);
}

Then I tried to get rid of the clip space position calculation so I can use the texture coordinate and depth directly. I did that by biasing the projection matrix (moving the coordinate system from -1.0 ... 1.0 to 0.0 ... 1.0) in C++ and then inverting it and supplying it as a uniform. The performance boost of doing that was small, but it's something. The final shader code looks like this:

vec3 calculate_view_position(vec2 texture_coordinate, float depth_from_depth_buffer)
{
    vec4 view_position = vec4(biased_inverse_projection_matrix[0][0] * texture_coordinate.x + biased_inverse_projection_matrix[3][0],
                          biased_inverse_projection_matrix[1][1] * texture_coordinate.y + biased_inverse_projection_matrix[3][1],
                          -1.0,
                          biased_inverse_projection_matrix[2][3] * depth_from_depth_buffer + biased_inverse_projection_matrix[3][3]);

    return(view_position.xyz / view_position.w);
}

The C++ matrix biasing code looks like this:

const MATH::FLOAT4X4 bias_matrix(0.5f, 0.0f, 0.0f, 0.5f,
                                 0.0f, 0.5f, 0.0f, 0.5f,
                                 0.0f, 0.0f, 0.5f, 0.5f,
                                 0.0f, 0.0f, 0.0f, 1.0f);   

MATH::FLOAT4X4 inverse_biased_projection_matrix = projection_matrix * bias_matrix; // Offsets the projection matrix' clip space coordinates from -1.0 - 1.0 to 0.0 - 1.0.
MATH::FLOAT4X4::invert(inverse_biased_projection_matrix);

glUniformMatrix4fv(uniform.id, 1, GL_FALSE, &inverse_biased_projection_matrix.elements[0][0]);

Depending on how you calculate your projection matrix, this might not work for you.

======== UPDATE ========
If you're linearizing your depth buffer or storing the view space z (like me now), you can do the following, which is the fastest possible way to reconstruct the view space position:

Vertex shader (can also be supplied as a uniform):

v_fov_scale.x = tan(fov_x / 2.0);
v_fov_scale.y = tan(fov_y / 2.0);
v_fov_scale *= 2.0; // Required to avoid the multiplication by 2.0 in the fragment shader at "vec2 P_ndc = vec2(1.0) - texture_coordinate * 2.0;".

Fragment shader:

vec3 calculate_view_position(vec2 texture_coordinate, float depth, vec2 scale_factor)  // "scale_factor" is "v_fov_scale".
{
    vec2 half_ndc_position = vec2(0.5) - texture_coordinate;    // No need to multiply by two, because we already baked that into "v_tan_fov.xy".
    vec3 view_space_position = vec3(half_ndc_position * scale_factor.xy * -depth, -depth); // "-depth" because in OpenGL the camera is staring down the -z axis (and we're storing the unsigned depth).
    return(view_space_position);
}

What this method does is, it basically computes a ray from the camera position to the far plane (in view space), which then gets scaled by the depth from the texture. If you aren't storing the view space Z, but just the normalized, linear depth, then you should multiply v_fov_scale with your depth range (zFar - zNear) and offset the coordinate to zNear in the fragment shader.