OpenGL approach to depth-testing-like combination of a large buffer of fragments

Question

I have a depth buffer and color buffer created by another effect. The buffer is more than double my screen size. It is not rendered from geometry, but as a whole it resembles a rendered scene with an ortho projection. I can't generate this data w/ any other projection. I'm trying to redraw the scene in two-point perspective. This means each fragment from the orthographic data will be skewed and translated based on the depth value.

If I render a fullscreen quad, each fragment will not know how or where to sample from the original buffer. It seems like I the sources have to assign themselves to destinations, rather than a destination gathering sources.

Impractical or incomplete solutions: - Generate a tri or point for every pixel in the source buffer, and actually rasterize them with depth testing. This is really slow, but it could be possible to generate the vertices in a compute shader or something? We're still dealing millions of tris for an average screen. I'm skeptical that this is the real solution. - Something with SSBO and atomicMax. It seems like you can only compare your actual data, so I would have to pack my colors with depth values somehow.

This seems like a fairly fundemental operation, so I'm hoping there is some canonical technique or opengl extention that does this sort of thing for me well. Even a name for this operation might help me find existing literature.

The final solution must be realtime for typical screen sizes. C++/OpenGL.

Answer 1

When I've done something similar (reprojecting colour-depth images from the Kinect 2 into the game's 3D space), I...

1. ...made a Mesh with a vertex for each texel in the input texture. (You can break this into multiple mesh chunks if you have more texels than your vertex limit allows)

2. ...rendered the mesh as a collection of Point primitives, rather than triangles.

3. ...sampled the source texture in the vertex shader and offset the scene position of the vertex according to the depth value in the image.

From there, the regular depth buffering takes care of overlapping fragments.

I can't claim this is the most efficient route, but it was good enough for us to maintain a realtime framerate with new input images streaming continuously from the Kinect sensor.