How can I implement "cut outs" for lighting in OpenGL?

Question

So, I'm working with OpenGL (I'm not exactly sure of the version), and I want to do an old-style lighting setup by essentially drawing a black rectangle over the screen, and drawing white circles over the areas that should be lit. I already have the rectangle and the circles, but I can't figure out how to make it so that the white circles are transparent when applied to the black areas. Any ideas? I tried to use blending modes, but it still evades me...

EDIT: Here's the code that I'm using (Python).

view_buf = Buffer(GL_INT, 4)
glGetIntegerv(GL_VIEWPORT, view_buf)
view = view_buf.to_list() if hasattr(view_buf, "to_list") else view_buf.list

glDisable(GL_DEPTH_TEST)

glDisable(GL_LIGHTING)

glEnable(GL_BLEND)

if logic.lights['smooth']:
    glEnable(GL_LINE_SMOOTH)

# # Make sure we're using smooth shading instead of flat
# glShadeModel(GL_SMOOTH)

# Setup the matrices

glMatrixMode(GL_PROJECTION)
glPushMatrix()
glLoadIdentity()
gluOrtho2D(0, view[2], 0, view[3])
glMatrixMode(GL_MODELVIEW)
glPushMatrix()
glLoadIdentity()

#glAlphaFunc(GL_GREATER, 0.5)

glColor4f(0.0, 0, 0, 0.75)

glBegin(GL_QUADS);                              # Draw Black screen
glVertex2d(0.0, 0.0);
glVertex2d(view[2], 0.0);
glVertex2d(view[2], view[3]);
glVertex2d(0.0, view[3]);
glEnd()

glColor4f(1, 1, 1, 1)

degtorad = 3.14159/180.0


glBlendFunc(GL_ZERO, GL_SRC_ALPHA)

def DrawCircle(x, y, radius, fill = 1, step = 11):

    glVertex2f(x, y)

    if fill:
        glBegin(GL_TRIANGLE_FAN)
    else:
        glBegin(GL_LINE_LOOP)

    for a in range(0, 360, step):
        angle = degtorad * a
        glVertex2f(x + (sin(angle) * radius), y + (cos(angle) * radius))

    glEnd()


#for layer in range(len(logic.lights['layers'])):
#   size = 
for light in logic.lights['points']:
    DrawCircle(light[0][0] * view[2], (1 - light[0][1]) * view[3], light[1], step = light[2])
    if logic.lights['smooth']:
        DrawCircle(light[0][0] * view[2], (1 - light[0][1]) * view[3], light[1], 0, light[2])

# Reset the state
glPopMatrix()
glMatrixMode(GL_PROJECTION)
glPopMatrix()

Answer 1

You can use the stencil buffer using a technique similar to the one described here for rendering to a non-rectangular viewport (as that is essentially what you are doing).

When you clear you scene, clear the stencil buffer to some fixed value (say zero). When you render the lit areas, configure the stencil mask to write a different value (say one) to the buffer. Then render your regular scene with the stencil function configured to accept only fragments where the stencil value is greater than 0.

This will cause only fragments within the lit areas to render. It's straightforward, but it has the (fairly severe) disadvantage of providing very hard edges to the lit regions.

The other options are to perform the entire effect in the pixel shader using a shader global for the position and radius of the light source (problematic for supporting more than one light source) or alpha blending precomputed textures into the scene (this is the most general method, easily supporting multiple light sources and smooth transitions into darkness).