I'm trying to create a visual effect where an arbitrary part of the main scene is either clipped out by or cropped to an arbitrary volume. That is to say, I might want to draw just the area of the scene inside the volume, enhanced by other elements, or only the area outside the volume, leaving empty an area I can fill in later.

The main scene is mostly a 10k-500k face surface (not solid). The clipping volume is an extruded polygon, usually 200-300 faces for the entire volume, not generally convex but easy to convexify if needed.

I've tried an approach basically identical to stencil shadow volumes, but where instead of drawing a lighter/darker version of the same scene, I just don't draw the pixels either in or out of the stencil area. This is about 85% of the way to the solution, except that (for the clip mode), pixels from the scene inside the clip volume that would normally occlude other parts of the model, but should be ignored, are left in the stencil.

Incorrect stenciling when clipping

The problem also seems eerily similar to constructive solid geometry. I would rather not solve it by manipulating the geometry itself, as manipulating 500k+ faces in-browser has not shown itself to be terribly performant (yes, I tried). There's a little bit of work on doing arbitrary CSG via stencils (like this) but it doesn't seem to have gotten much attention in the last decade.

In my ideal case, I'd be able to use some set of techniques to individually identify regions of the viewport that contain:

  1. Pixels outside the clip volume, even if they would normally be occluded by pixels inside the clip volume
  2. Pixels that the mesh inside the clip volume would occupy, even if normally occluded by other pixels outside the clip volume

My target is WebGL, but happy to look at plain OpenGL techniques that may be applicable, or those enabled by WebGL2.

  • \$\begingroup\$ where's the game? \$\endgroup\$ – gman Apr 17 '17 at 11:05
  • \$\begingroup\$ I don't think I understand exactly what's going wrong with your stencil approach. Is it that pixels that should have been skipped (those between the dashed lines) are still occluding pixels behind them? \$\endgroup\$ – Victor T. Apr 17 '17 at 14:39
  • \$\begingroup\$ @victor-t Yes, exactly. Tried [poorly] to highlight the red area that should be drawn but isn't. \$\endgroup\$ – John Laxson Apr 18 '17 at 0:16
  • \$\begingroup\$ That seems strange, it suggests that your shader is writing pixels to the z buffer even when the stencil is rejecting them from the color buffer. How do you throw out the stenciled-out pixels? \$\endgroup\$ – Victor T. Apr 18 '17 at 15:01
  • \$\begingroup\$ The process I follow now is: * Disable color write, enable depth write * Render full scene * Disable depth test, color write * ctx.stencilOpSeparate(ctx.BACK, ctx.KEEP, ctx.DECR_WRAP, ctx.KEEP); * ctx.stencilOpSeparate(ctx.FRONT, ctx.KEEP, ctx.INCR_WRAP, ctx.KEEP); * Render the clip volume * Clear depth, disable stencil write, enable color write, depth test/write * Stencil test for zero, should be the areas outside the clip volume * Render full scene again \$\endgroup\$ – John Laxson Apr 18 '17 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.