I hope I could get some help here in finally resolving this problem of mine re: shadow casting with alpha textures.

Here's a screenshot on what I am hoping for as a result:

enter image description here

A friend of mine just sent this shader program to help out but I still can't get it to work properly. The alpha textures such as leaves still leave a rectangular shadow. I'm using a custom built Ogre based engine but overall the material and compositor system is still the same.

Attachment: Alpha_Shader.zip

Any help would be greatly appreciated.



1 Answer 1


To do this, you must sample your alpha texture in the depth buffer creation fragment shader, which is ShadowCasterFP in your code. When you sampled the texture, you should discard pixels below a certain alpha level, or do a clip:

if( color.a<0.1 ) 

clip( color.a<0.1?-1:1 );
//where color is your sampled texture at the current fragment

These two will do the same, which is not outputting any color to your rendertarget/depthmap.

EDIT: You'd have to insert the above code (either the discard or the clip) to your MVSMShadows.hlsl to the beginning of the ShadowCasterFP function. But you also have to sample a texture resource and a sampler there which you must declare and upload to the GPU beforehand. You can declare a texture and a sampler like this:

Texture2D<float4> myTexture : register(t0); //texture bound to the first texture slot
SamplerState mySampler : register(s0); //sampler unit bound to the first sampler slot

Then you must sample that texture while creating the shadow map:

float4 color = myTexture.Sample(mySampler,In.texCoords);

So you have your fragment color which contains an alpha. You check that alpha and discard if it is too low with clip or discard.

  • \$\begingroup\$ Hey, thank you for this info. Honestly, I don't have any background in shader programming so could you kindly provide a revised code based on the files that I sent? I'm not sure if they are working or where to place the corresponding additions that you mentioned. Perhaps via Pastebin? Thanks again. \$\endgroup\$
    – scottyp
    Commented Jul 19, 2014 at 14:15
  • \$\begingroup\$ Added some info in the edit. \$\endgroup\$ Commented Jul 20, 2014 at 20:48
  • \$\begingroup\$ Thanks a lot János! I think I'm now getting somewhere but I'm not sure where to place the texture and sampler line you mentioned above, is it still under ShadowCasterFP function? And the float4 color.. goes into ShadowReceiverFP right? I don't know where to exactly place the last 2 codes but so far I got this on MVSMShadows.hlsl pastebin.com/0shkvLqu and my update Tree.material is this pastebin.com/jTahK13i \$\endgroup\$
    – scottyp
    Commented Jul 21, 2014 at 11:38
  • \$\begingroup\$ The texture and sampler declarations go outside of the function, like you are declaring uniforms, the color goes into the function before the clip. \$\endgroup\$ Commented Jul 21, 2014 at 12:04
  • \$\begingroup\$ Ok thanks. I now got this snippet on my MVSMShadows.hlsl pastebin.com/xWanPsCX I still can't get the alpha texture shadows to work.. :( \$\endgroup\$
    – scottyp
    Commented Jul 21, 2014 at 12:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .