I find two methods to produce cubemap for shadow both using the same geometry shader to draw each of the 6 faces. The first one writes in the depth buffer only and is used like this (really I thanks the guy having posted this code elsewhere (https://community.monogame.net/t/view-matrix-for-point-light-shadows/14548)):
float3 txDir = Input.WPos.xyz - LightPos[2].xyz;
float3 LtoP = normalize(txDir);
float Closet = -1;
int FIndex = 0;
for (int f = 0; f < 6; f++)
{
float r = dot(LtoP, FDir[f]);
if ( r > Closet )
{
Closet = r;
FIndex = f;
}
}
float4 LPos = mul(Input.WPos, LightViewProjCube[FIndex]);
LPos.z/=LPos.w;
D = txShadowCubeMap.Sample(samWrap, txDir).r;
Shadow = (D + 0.001f < LPos.z)?0:1;
as one can see you need some job to retrieve the correct face index and its light matrix to recalculate the position but this code works well.
the alternative solution from learngl tutorial use the same geometry shader but writes his own depth value in the depth buffer (I use a render target R16F in DX11 for this) in a 0..1 range via 1/farplane
float PS_CubeMap(PS_CUBEMAP_IN Input): SV_TARGET
{
return length(Input.Pos.xyz - LightPos[2])*0.001f;//0.001 is 1/farplane
}
then they use it like this:
float3 txDir = Input.WPos.xyz - LightPos[2].xyz; //2nd pointlight (yellow)
D = txShadowCubeMap.Sample(samClamp, txDir ).r;
if ( D < 1 ) //rendertarget red chanel cleared to 1. So 1 means nothing was drawn.
Shadow = (length(txDir) > D*1000)?1:0;//*1000 restores the real distance
My problem here is that if the second method is 5-10% faster because less pixel operations it introduces a zbias problem for object being both shadow caster and receiver I can't figure out how to solve (see picture). I've tried to "push away" the distance value when generating the map or various additional bias when using the texture, or back/front cull for rendering the shadow, various sampler. No improvements. Of note all my matrices have the same 1/1000 near/farplane values. Is there something in this learngl code adaptation I'm missing?