Sorry for the bad title; I don't know how else to explain this.
I have a class called a Container
. An instance of Container
has other elements "inside" it. These elements have independent positions that are relative to the position of the container.
So if I move the container, the elements inside of it are moved along with it; but keep their distance relative to each other.
This Container
is therefore like a window through which you can view it's contents.
So, I would like to use a Stencil buffer for this. I have the following code:
glEnable(GL_STENCIL_TEST);
glClear(GL_STENCIL_BUFFER_BIT);
glStencilMask(0xFF); // everything is written to the buffer
glStencilFunc(GL_ALWAYS, 1, 0xFF);
glStencilOp(GL_KEEP,GL_KEEP,GL_REPLACE);
glColorMask(GL_TRUE,GL_FALSE,GL_FALSE,GL_FALSE);
glBindVertexArray(m_screenQuad->GetVAO());
glDrawElements(GL_TRIANGLES,m_screenQuad->GetMesh()->GetIndices()->size(),GL_UNSIGNED_INT,0);
glBindVertexArray(0);
glStencilFunc(GL_EQUAL, 1, 0xFF);
glStencilOp(GL_KEEP,GL_REPLACE,GL_REPLACE);
glColorMask(GL_TRUE,GL_TRUE,GL_TRUE,GL_TRUE);
// draw the things that are visible through this window. The window is defined by m_screenQuad
glDisable(GL_STENCIL_TEST);
This works great.
The problem I'm having arises when one of the things that are visible through the window, are also another window.
In the above example you can see that I didn't mask out the red channel, this was to reveal that when the bounds of an "interior" container went outside the edge of he exterior container, the whole stencil doesn't work as planned.
Here's some screen shots with the above code (again, note that the red it to show the different bounds being drawn by the m_screenQuad
objects).
Here's another image where the object inside the first Container
is a subsequent Container
. Each calls the same code above, with the second one being called at the // draw the things that are visible through this window. The window is defined by m_screenQuad
line.
As you can see, the bounding m_screenQuad
being drawn in the same way by the internal Container
overlaps with the external Container
s m_screenQuad
and effectively extends the Stencil Mask further than it should.
How do I separate these two different Stencil tests so that they don't affect each other? Consider that the internal Container
may be smaller than the external one, so it would need to truncate/cut it's internal contents as well.