moving glDepthMask into a shader

Question

Can a fragment shader make per-fragment decisions on whether the fragment updates the depth buffer or not, even if the fragment is not discarded and the color is written?

Answer 1

I don't think so. If you don't write into gl_FragDepth, OpenGL writes the interpolated depth. There is no magic "discard" value you could write into gl_FragDepth and like you yourself say, discard also discards colors.

And as a side note I'm also sure to have read, that if you write into gl_FragDepth in one dynamic branch of a shader, you have to write into it in all branches, otherwise the results are undefined. But maybe this has changed on newer hardware.

EDIT: Maybe you can use a (though quite strange) workaround using ARB_shader_stencil_export extension (or perhaps some newer core version supporting it). In the first pass disable depth update (depth mask false) and write a special stencil value (maybe 1) for the fragments you want to write the depth for and another (0) for the ones you don't. Then in the second pass you draw the scene/objects/whatever again enabling depth write and configuring the stencil test to only pass where the stencil buffer is set. So only the depth of the fragments you (dynamically) selected are written. But the two-pass nature of this might bring some problems whith "underdraw" that shouldn't be, as in the first pass no fragment writes any depth value.