Update:
I am getting slightly better results by modifying the shader so that it sets a flag on entering a obscured region, and then only sets the fragment to black if the ray casting exits the obscured region. The bordering light is a little less noticeable this way. Additionally objects behind other objects will now be correctly obscured as long as they are not touching.
However as you can see in the image, I have noticed a new problem. The shadow becomes very choppy at the back due to the way I am doing raycasting.
Original:
I am currently trying to render shadows in a 2d game. I use the following shader to do so:
#version 120
varying vec2 vTexCoord;
uniform vec3 u_lightColor;
uniform sampler2D u_texture;
float drawShadow(){
vec2 loc;
vec2 norm = normalize(vTexCoord - vec2(.5,.5)); // Pointing from center to point
vec4 smp;
float dst = 0.0;
// For now draw all casters
if(texture2D(u_texture, vTexCoord - dst * norm).a > 0)
return 1.0;
for(int i = 0; i < 256; i++){
loc = vTexCoord - dst * norm;
if (length(loc-vec2(.5,.5)) < 1.0/256.0)
break;
smp = texture2D(u_texture, loc);
if(smp.a > 0){
return 0.0;
}
dst += 1.0/256.0;
}
return 1.0;
}
void main() {
float dist = length(vec2(.5,.5) - vTexCoord.xy);
float shad = drawShadow();
float intensity = 1-dist*2.0;
gl_FragColor = vec4(u_lightColor, 1.0)*vec4(shad*vec3(intensity), shad);
}
The light is being rendered at the center of the screen, and the texture being drawn is one where any shadow caster is drawn as a non transparent sprite.
The trouble is because of the way I am sending out the rays I end up with the following:
Where the shadows don't hug the edges of the sprites. Also, I would like to somehow obscure shadow casters if they are in the shadow of another shadow caster, but not if they are the first caster the light hits.
shad
value of 0.0 creating a 0 alpha color value, but also multiplying it by your light color value guaranteeing you are only going to see your background color when you actually get a shadow with the current code. Additionally you appear to only create shadows when there exists sample alpha and using that to determine if you are actually inside a given texture, meaning all we see should be background color. You're leaving out a lot of information \$\endgroup\$