3
\$\begingroup\$

I'm working on building out a 2.5D engine and having massive problems getting my shadows working. I'm at a point where I'm VERY close.

So, let's see a picture to see what I have:

Screenshot

As you can see above, the image has lighting -- but the shadow map is displaying incorrectly. The shadow map is shown in the bottom left hand side of the screen as a normal 2D texture, so we can see what it looks like at any given time.

If you notice, it appears that the shadows are generating backwards in the wrong direction -- I think. But the problem is a little more deep -- I'm just plotting the shadow onto the screen, which I know is wrong -- I'm ignoring the actual test to see if we NEED to show a shadow.

The incoming parameters all appear to be correct -- so there has to be something wrong with my shader code somewhere.

Here's what my code looks like:

VERTEX:

    uniform mat4 LightModelViewProjectionMatrix;
    varying vec3 Normal;            // The eye-space normal of the current vertex.
    varying vec4 LightCoordinate;   // The texture coordinate of the light of the current vertex.
    varying vec3 LightDirection;    // The eye-space direction of the light.

    void main()
    {   
        Normal = normalize(gl_NormalMatrix * gl_Normal);    
        LightDirection = normalize(gl_NormalMatrix * gl_LightSource[0].position.xyz);
        LightCoordinate = LightModelViewProjectionMatrix * gl_Vertex;
        LightCoordinate.xy = ( LightCoordinate.xy * 0.5 ) + 0.5;

        gl_Position = ftransform();

        gl_TexCoord[0] = gl_MultiTexCoord0;
    }

FRAGMENT:

    uniform sampler2D DiffuseMap; 
    uniform sampler2D ShadowMap;

    varying vec3 Normal;            // The eye-space normal of the current vertex.
    varying vec4 LightCoordinate;   // The texture coordinate of the light of the current vertex.
    varying vec3 LightDirection;    // The eye-space direction of the light.

    void main()
    {   
        vec4 Texel = texture2D(DiffuseMap, vec2(gl_TexCoord[0]));

        // Directional lighting

        //Build ambient lighting
        vec4 AmbientElement = gl_LightSource[0].ambient;

        //Build diffuse lighting
        float Lambert = max(dot(Normal, LightDirection), 0.0); //max(abs(dot(Normal, LightDirection)), 0.0);

        vec4 DiffuseElement = ( gl_LightSource[0].diffuse * Lambert );

        vec4 LightingColor = ( DiffuseElement + AmbientElement );

        LightingColor.r = min(LightingColor.r, 1.0);
        LightingColor.g = min(LightingColor.g, 1.0);
        LightingColor.b = min(LightingColor.b, 1.0);
        LightingColor.a = min(LightingColor.a, 1.0);

        LightingColor *= Texel;


        //Everything up to this point is PERFECT

        // Shadow mapping
        // ------------------------------
        vec4 ShadowCoordinate = LightCoordinate / LightCoordinate.w;

        float DistanceFromLight = texture2D( ShadowMap, ShadowCoordinate.st ).z;

        float DepthBias = 0.001;
        float ShadowFactor = 1.0;

        if( LightCoordinate.w > 0.0 )
        {
            ShadowFactor = DistanceFromLight < ( ShadowCoordinate.z + DepthBias ) ? 0.5 : 1.0;
        }

        LightingColor.rgb *= ShadowFactor;

        //gl_FragColor = LightingColor;


        //Yes, I know this is wrong, but the line above (gl_FragColor = LightingColor;) produces the wrong effect
        gl_FragColor = LightingColor * texture2D( ShadowMap, ShadowCoordinate.st );
    }

I wanted to make sure the coordinates were correct for the shadow map -- so that's why you see it applied to the image as it is below. But the depth for each point seems to be wrong -- the shadows SHOULD be opposite (look at how the image is -- the shaded areas from normal lighting are facing the opposite direction of the shadows).

Maybe my matrices are bad or something going in? They're isolated and appear to be correct -- nothing else is going in unusual. When I view from the light's view and get the MVP matrices for it, they're correct.

EDIT: Added an image so you can see what happens when I do the correct command at the end of the GLSL:

Screenshot2

That's the image when the last line is just glFragColor = LightingColor;

Maybe someone has some idea of what I screwed up?

\$\endgroup\$
  • \$\begingroup\$ Have you resolved your problem? I have the similar or the same problem now. No "right" shadow, but many dark points or little spots. \$\endgroup\$ – user20262 Sep 27 '12 at 7:48
  • \$\begingroup\$ For me, I actually dropped work on the project, but I felt that I was really close and just needed to flip the end image -- which a colleague of mine in Germany said happens occasionally (and needs to be done in Direct X). I would output your shadow map and depth map and ensure everything is correct -- and if so, start trying to figure out if you need to adjust your coordinates. \$\endgroup\$ – Locke Sep 27 '12 at 18:11
  • \$\begingroup\$ Thanks for answer, Locke. You are writing now: "..shadow map and depth map..". But it is the same, I mean. You wrote at Aug 30: "ShadowMap is actually a DepthMap". I don't understand.. \$\endgroup\$ – user20303 Sep 28 '12 at 9:10
1
\$\begingroup\$

I think the error is somewhere here:

    float DistanceFromLight = texture2D( ShadowMap, ShadowCoordinate.st ).z;

A shadowmap does not produce the distance from a light but produces if the light can 'see' the area we are shading now. This is why the 'wrong' line works. It is fragmentColor * 0 if the light cant see the area and fragmentColor * 1 when it can.

\$\endgroup\$
  • \$\begingroup\$ Well, the ShadowMap is actually a DepthMap, when sending in -- so it is that depth. Even if that is the case, if you look at the shadow, it's on the wrong side -- it should be on the opposite side of where the light is at. While I'd love to be able to just do a 1 - s and 1 - t to flip it, it's only a workaround -- not an actual fix. There has to be something else wrong, but thank you for your comment so far! \$\endgroup\$ – Locke Aug 25 '12 at 13:19
  • \$\begingroup\$ Urgh you're absolutely right. It's been a while since I've used shadowmaps and it shows. In the picture I still can't see why the shadows are wrong. The shadows are correctly at the dark side of the mountains. \$\endgroup\$ – Roy T. Aug 25 '12 at 14:41
  • \$\begingroup\$ It looks like the actual ShadowMap is flipped -- the shadows are not actually on the dark side of the mountains. They're on the "light" side of the mountains, so it seems like I need to flip its axis to get the correct result. Apparently this is common in Direct X implementations. Regardless, thanks for your help! Hopefully that will fix it. HOPEFULLY. /crosses fingers \$\endgroup\$ – Locke Aug 30 '12 at 16:51

Not the answer you're looking for? Browse other questions tagged or ask your own question.