# How to keep a square in front of moving camera?

I'm currently attempting to create a Lens of Truth effect like what we find in Zelda: Ocarina of Time:

The Lens of Truth allows [the player] to see things that are not normally visible to the naked eye. It can show fake walls and floors, and invisible objects and enemies. (Source: Zelda wiki)

I'm attempting it by using a square frame that is positioned in a scene and an object that should only be visible when viewed through that frame.

The way I tried it is, I defined the square vec4s like this and transform it using the world, view and projection matrix:

vec4 fc[4];
fc[0]=vec4(0,0,0,1);
fc[1]=vec4(1,0,0,1);
fc[2]=vec4(1,1,0,1);
fc[3]=vec4(0,1,0,1);
for(int i=0;i<4;i++){
fc[i].xyz =  vec4( (worldMat * fc[i]) ).xyz;
fc[i].w=1.0;
fc[i] = pass.Proj*(pass.View*fc[i]);
}


In theory I should now get the positions of these 4 points in screen space relative to the position of the object.

Now I should just be able to do this:

if(!inside(inPs.glPosition.xy,fc[0].xy,fc[1].xy,fc[2].xy,fc[3].xy) ){
}


to clip the fragment if it is outside the quad of those points (inside is function by me, the problem is not here), since I basically figure out the screen space coordinates of the square frame.

But if I run the thing the coordinates of the square corners move all over the place.

If I debug the thing by displaying the position of those points on the object, they don't move at all when I translate the camera, but only when I rotate it, when It should logically be the other way around. Imagine a sheet with a ligthbulb behind it. If you stand still, the position of the lamp on the sheet doesn't change, but when you move around, the light will be on other places of the sheet. I have a feeling, there's something wrong with my thought process here, but I can't figure out why.

• There's no single question sign in this post. Please clarify, what is the question of this question - is it about stencil/mask or is it about transformation of the rect? Feb 6, 2017 at 5:21
• I have edited question title and moved your answer into answers, as it should be. Feb 7, 2017 at 5:24

I'm not sure what's wrong with your math, but you're tackling this the wrong way. The way to do the lens of truth is with a stencil buffer. Basically you mark the pixels that the lens is covering then draw the hidden objects only on the marked pixels.

1. Draw your scene as normal except for the lens and hidden objects.
2. Disable color writes and depth writes and enable stencil writes.
3. Draw the polygons for just the glass of the lens and not the border.
4. Enable color writes, depth writes, and stencil test and disable stencil updates. Also create a clip plane parallel to the surface of the lens to hide objects that are between your eye and the lens.
5. Draw all the hidden objects. They will only be drawn on the pixels that the lens touched.
6. Disable stencil test and clip plane. (Your render settings are back to normal at this point. )
7. Draw the lens and frame, this time it will actually draw.

Don't forget to clear your stencil buffer for the next frame.

• Did you read the questions' ending? "If I debug .. they don't move at all when I translate the camera, but only when I rotate it, when It should logically be the other way around" Feb 6, 2017 at 5:14

Quote from user2741831:

If anyone finds this thread, the answer is to calculate gl_position.xy/gl_position.w to get 2D screen coordinates