Could someone explain why my world reconstructed from depth position is incorrect?

I am attempting to reconstruct the world position in the fragment shader from a depth texture. I pass in the 8 frustum points in world space and interpolate them across fragments and then interpolate from near to far by the depth:

``````highp float depth = (2.0 * CameraPlanes.x) / (CameraPlanes.y + CameraPlanes.x - texture( depthTexture, textureCoord ).x * (CameraPlanes.y - CameraPlanes.x));

// Reconstruct the world position from the linear depth
highp vec3 world = mix( nearWorldPos, farWorldPos, depth );
``````

CameraPlanes.x is the near plane CameraPlanes.y is the far.

Assuming that my frustum positions are correct, and my depth looks correct, why is my world position wrong?

(My depth texture is of format GL_DEPTH_COMPONENT32F if that matters)

Thanks! :D

Update:

Screenshot of world position http://imgur.com/sSlHd

So you can see it looks nearly correct. However as the camera moves, the colours (positions) change, which they shouldnt.

I can get this to work, if I do the following:

Write this into the depth attachment in the previous pass:

``````gl_FragDepth = gl_FragCoord.z / gl_FragCoord.w / CameraPlanes.y;
``````

and then read the depth texture like so:

``````depth = texture( depthTexture, textureCoord ).x
``````

However this will kill the hardware z buffer optimizations.

-
In what way is your world position wrong? – doppelgreener May 22 '12 at 11:12
what should it be doing, and what is it doing? can you give possibly a more concrete example. – gardian06 May 22 '12 at 13:01
Updated, thanks for looking~ – yuumei May 22 '12 at 13:18
Have you compared your approach to the `VSPositionFromDepth` function here? That function is less optimized, but simpler (requires only the inverse projection matrix), so it should be easier to get working. – Nathan Reed May 22 '12 at 16:39
Hm I could just use another render to texture as in the example you provided. However if I can use the depth buffer directly it will save the memory of another depth texture. I'm not sure if I can however. – yuumei May 23 '12 at 8:30

yuumei, I do not believe you are accounting for the perspective divide in your calculations. You are correctly determining the depth in the picture plane in your first equation, but then using a simple linear interpolation (mix()).

12.070 Why is there more precision at the front of the depth buffer?

After the projection matrix transforms the clip coordinates, the XYZ-vertex values are divided by their clip coordinate W value, which results in normalized device coordinates. This step is known as the perspective divide. The clip coordinate W value represents the distance from the eye. As the distance from the eye increases, 1/W approaches 0. Therefore, X/W and Y/W also approach zero, causing the rendered primitives to occupy less screen space and appear smaller. This is how computers simulate a perspective view.

This is why when you divide by the fourth coordinate in your alternative implementation the results are correct.

Incidentally, whenever I calculated the world space position from the depth buffer the method I used was to retrieve the picture plane depth (as you do), then project a ray with the camera properties for each pixel like so:

``````//Get depth value to resolve
depthValue d = depthValues[i];

//Compute clip-space coordinate of pixel
float2 screenspace = ( (d.screenposition) / ((float2)(width,height)) ) * 2.0f - 1.0f;

//Get direction of ray from camera through pixel
float3 ray_direction = normalize(camera_forward.xyz + (camera_right.xyz * screenspace.x) - (camera_up.xyz * screenspace.y));

//Reconstruct world position from depth: depth in z buffer is distance to picture plane, not camera
float distance_to_camera = d.depth / dot(ray_direction, camera_forward.xyz);
float3 world_position = camera_position.xyz + (ray_direction * distance_to_camera);

positions[i] = (float4)(world_position, 1);
``````

This is not as 'short' as some other methods to be found on-line; but this always worked for me and I cannot necessarily say the same of them!

Good Luck!

-
sebf, are your depth values stored as z/w? Because I would like to be able to use the depth buffer directly and not add another depth fbo. – yuumei May 23 '12 at 16:27
@yuumei; I believe so as they are written by the pipeline which includes the perspective divide. My implementation was in OpenCL but there should be no need to use an interim buffer, you just need to pass in the camera properties which were used to generate the picture plane your depth values are in. – sebf May 23 '12 at 21:52
So it turned out that the format of the depth buffer was the problem. But this explanation helped, thanks :3 – yuumei May 30 '12 at 10:49
I took a look at this problem again recently as I did not solve it. I found that this method did not work as I am reading directly from the depth buffer so get back values that are post perspective transform. I believe you must be storing your depth values in a linear format. – yuumei Jul 2 '13 at 11:02

After a while I came back to this question. I was using the OES_depth_texture extension. I found that the answer was explained here very well: http://www.opengl.org/wiki/Compute_eye_space_from_window_space#From_XYZ_of_gl_FragCoord

I have used this method with success. Unfortunately all other methods did not work. At the end you get a vec4 eyePos which you can multiply by the inverse view matrix to get to world space.

-