# Light direction not being calculated correctly

I'm trying to implement deferred lighting in my game, and have run into an issue with normals.

The image above shows the lighting being applied to a tile map. The yellow shade represents the direction of the light at that pixel. The character has a green normal so when he is below the light as he is in the screenshot, he doesn't get lit. I was working through the problem of the character always being lit, and it seems to be because the light direction isn't calcualted correctly. I expect the light direction color to be a rainbow of colors, however the yellow means that the light is always point down X and Y axis instead of radiating out in a circle. I'm certain that worldPos is coming out correct because the diffuse/normal colors are being displayed correctly.

#version 330

uniform sampler2D diffuseTexture;
uniform sampler2D normalsTexture;
in vec2 TexCoords;

out vec4 outColor;

void main(void) {
vec2 resolution = vec2(200, 200);
float distance = length(vec3((TexCoords*resolution) - (resolution / 2),
0.0));
float constantAttenuation = 1.0;
float linearAttenuation = 0.1;

float attenuation = 1.0 / (constantAttenuation +

vec3 worldPos = gl_FragCoord.xyz / vec3(1024, 768, 1.0);
vec3 lightDir = normalize(vec3(50,50,10) - worldPos);

float diffuseFact = dot(normalize(texture2D(normalsTexture, worldPos.xy).rgb), lightDir);

vec4 diffuse = vec4(0.0);

if(diffuseFact > 0.0) {
diffuse = vec4(texture2D(normalsTexture, worldPos.xy).rgb, 1.0);
diffuse = diffuse * diffuseFact * 10;
}

outColor = vec4(lightDir, 0.5) + (diffuse*attenuation);
}

• Two things pop out at me: Attenuation is usually relative to the light position, but here you're using distance, which is simply based on the screen coordinate of the pixel. Second, what's the format of normalsTexture? I'd expect you to have encoded normals (maybe (normal vector) * 0.5 + (0.5, 0.5, 0.5)?) and then decode them in this shader (i.e. (encoded normal) * 2 - (1, 1, 1)). – Victor T. May 14 '14 at 17:56
• I'm rending the light out via a rectangle, and calculating the light from the center of it. This is so I don't have to run the fragment shader on every pixel on the screen for every light. The format of normalsTexture is a RGB texture. Right now I'm just assigning specific colors to the tile map and the sprite for testing. The sprite is full green and the map is full white in the screenshot – Ryan Capote May 14 '14 at 18:01
• I would advise against using magic numbers in your transformation from window-space (gl_FragCoord.xyzw) to world-space. You are also neglecting to consider the division by w that occurs between clip-space and window-space. Window-space w is 1.0/clip_w and this is done so that you can properly undo the original division by w to reverse the transformation. – Andon M. Coleman May 14 '14 at 18:05
• @AndonM.Coleman Definitely, I'll be replacing the magic numbers with varibles once I get around to cleaning up the code. Right now I just want to get it functioning. I'm not certain what you mean by your second point. Since I'm using orthographic projection, w isn't included in the calculation – Ryan Capote May 14 '14 at 18:21
• Your question mentions nothing about what kind of projection you are using. To handle the general case, you must always divide by w. It may be a no-op in an orthographic projection, however, even then there are situations where w is not 1.0; the only thing an orthographic projection guarantees is that w does not vary with distance down the z-axis. Speaking of which, worldPos.z is always between 0.0 and 1.0 assuming default depth range. You are computing the direction from (50,50,10) to a location that is (...,...,[0,1]); lightDir will always point down the Z-axis. – Andon M. Coleman May 14 '14 at 18:26