# How can I implement "cut outs" for lighting in OpenGL?

So, I'm working with OpenGL (I'm not exactly sure of the version), and I want to do an old-style lighting setup by essentially drawing a black rectangle over the screen, and drawing white circles over the areas that should be lit. I already have the rectangle and the circles, but I can't figure out how to make it so that the white circles are transparent when applied to the black areas. Any ideas? I tried to use blending modes, but it still evades me...

EDIT: Here's the code that I'm using (Python).

view_buf = Buffer(GL_INT, 4)
glGetIntegerv(GL_VIEWPORT, view_buf)
view = view_buf.to_list() if hasattr(view_buf, "to_list") else view_buf.list

glDisable(GL_DEPTH_TEST)

glDisable(GL_LIGHTING)

glEnable(GL_BLEND)

if logic.lights['smooth']:
glEnable(GL_LINE_SMOOTH)

# # Make sure we're using smooth shading instead of flat
# glShadeModel(GL_SMOOTH)

# Setup the matrices

glMatrixMode(GL_PROJECTION)
glPushMatrix()
glLoadIdentity()
gluOrtho2D(0, view[2], 0, view[3])
glMatrixMode(GL_MODELVIEW)
glPushMatrix()
glLoadIdentity()

#glAlphaFunc(GL_GREATER, 0.5)

glColor4f(0.0, 0, 0, 0.75)

glBegin(GL_QUADS);                              # Draw Black screen
glVertex2d(0.0, 0.0);
glVertex2d(view[2], 0.0);
glVertex2d(view[2], view[3]);
glVertex2d(0.0, view[3]);
glEnd()

glColor4f(1, 1, 1, 1)

degtorad = 3.14159/180.0

glBlendFunc(GL_ZERO, GL_SRC_ALPHA)

def DrawCircle(x, y, radius, fill = 1, step = 11):

glVertex2f(x, y)

if fill:
glBegin(GL_TRIANGLE_FAN)
else:
glBegin(GL_LINE_LOOP)

for a in range(0, 360, step):
angle = degtorad * a
glVertex2f(x + (sin(angle) * radius), y + (cos(angle) * radius))

glEnd()

#for layer in range(len(logic.lights['layers'])):
#   size =
for light in logic.lights['points']:
DrawCircle(light[0][0] * view[2], (1 - light[0][1]) * view[3], light[1], step = light[2])
if logic.lights['smooth']:
DrawCircle(light[0][0] * view[2], (1 - light[0][1]) * view[3], light[1], 0, light[2])

# Reset the state
glPopMatrix()
glMatrixMode(GL_PROJECTION)
glPopMatrix()

• Without seeing your code, it's hard to tell. The blend modes should do what you need, but perhaps your implementation is bugged or incomplete. Oct 20 '11 at 7:56
• Most probably you need to look at glBlendFunc and it's parameters Oct 20 '11 at 9:34
• Huh, okay. Lemme see if I can post the code... Oct 20 '11 at 14:09
• Okay, so after doing a little research, it would appear that I need to render the lights and darkness to a texture first, and then draw that over my actual scene... Oct 20 '11 at 14:25

## 1 Answer

You can use the stencil buffer using a technique similar to the one described here for rendering to a non-rectangular viewport (as that is essentially what you are doing).

When you clear you scene, clear the stencil buffer to some fixed value (say zero). When you render the lit areas, configure the stencil mask to write a different value (say one) to the buffer. Then render your regular scene with the stencil function configured to accept only fragments where the stencil value is greater than 0.

This will cause only fragments within the lit areas to render. It's straightforward, but it has the (fairly severe) disadvantage of providing very hard edges to the lit regions.

The other options are to perform the entire effect in the pixel shader using a shader global for the position and radius of the light source (problematic for supporting more than one light source) or alpha blending precomputed textures into the scene (this is the most general method, easily supporting multiple light sources and smooth transitions into darkness).

• Hmm. Thanks, I'll check it out. EDIT: Heh, alpha blending precomputed textures is how I got it to work in the first place! :D But I'm using a game engine, and wanted to get rid of the game engine frame buffer object that I was rendering to. :I Maybe I'll see about GLSL shaders. Thanks for the info! Oct 20 '11 at 16:26