What methods are there to discard points on one side of a surface but not the other?

Goal: Given a surface mesh (terrain), discard all points above a given elevation relative to the terrain. The terrain can change quite often due to tiling and LODs, so the point culling will need to be dynamic and have good performance. The camera in our scene is free floating, so no assumptions can be made in that respect.

Here is a side view of the problem:

My initial attempt was to draw the terrain, translated to the given elevation, only to the depth buffer. Then, when rendering the point cloud, discarding points that fail the depth test. There are a couple problems with this method though: I have to determine which side of the translated terrain the camera is on and, if above, flipping the depth test. Also, if the camera is looking at the terrain edge-on, points will be discarded that shouldn't be.

Any ideas are welcome, and let me know if I need to make clarifications regarding the task.

If you have a height map for your terrain (or can generate one for culling purposes), you can write a vertex shader for your point cloud that samples this heightmap at the vertex's worldspace position to compute the culling altitude at that point.

Then you can abort points above the culling altitude with this line:

projectedVertex.x /= step(worldspaceVertex.y, cullingAltitude);


when the vertex is above the culling altitude, this puts an infinity or NaN into its projected position, which will block it and any triangles / rasterized points depending on it from being drawn.

If you're using point topology you could also just push the vertex's projected position past the far plane. The trick above is handy for triangles that may straddle the culling threshold, because it aborts the whole triangle rather than distorting it.

Your technique is mostly correct. You need to tweak it to be more like basic shadow mapping. Render the terrain to texture, depth buffer only, using a top-down orthographic projection (this corresponds to the light source in shadow mapping). Use what ever lod/tile geometry you want to clip against. The bounds of this camera represent the area you’re interested in. Then, in your shader, check if the point + offset is “shadowed”.

This should work even if the camera is under the terrain. If you want the camera position to affect the results then calculate what side the camera is on once per frame and boil it down to a +-1. Use that in your distance equation.