Project rendering onto a polygon in the scene

Question

I have a function that renders a simple OpenGL scene. I want to display this scene on a polygon within another scene, without the function having to know about it. For example, to display the scene on the red polygon indicated:

I can set up the projection and modelview matrices to render to a small rectangle on the screen, but that doesn't work in a case like the image shown, where the target polygon is transformed and is no longer a rectangle on the screen. Is it possible to render to a non-rectangular target like this? The target is defined by four vertices, and I'm just not sure how to put those into the matrices.

I am aware of VBO/FBO functions that can render the sub-scene to a texture, which could then be mapped onto the polygon. However, this would be a bit more work as I'd have to add support for these functions to LuaGL, and I get the feeling it'd be slower than simply altering the display matrices - if it can even be done that way.

Answer 1

You can use the stencil buffer to clip an arbitrary shape out. However, you still need to compute the orientation of that red polygon, and figure out what pixels it occupies, so you still have to use an FBO in order to get that stencil buffer each frame.

1) Render the small red polygon (to an FBO still) and create a stencil buffer from the pixels it covered

2) Orient the camera as you like and render the scene with that stencil buffer you just created active

Using an FBO (recommended, easier, more flexible, "standard" way to do this kind of thing)

The way you would do this is:

1) Create an FBO with a texture2D as a backing

2) Render the scene (to the texture)

3) Now render your scene to the screen FBO, and render the texture you just created to the polygon you want it to display on

This is how you get "mirrors" to appear in games too.

Answer 2

You can use glReadPixels to read pixels from the frame buffer - explanation page
Hope this helps!

Answer 3

Doesn't LuaGL support any buffers? These can be normal renderbuffers, if FBO are unavailable. Anyway, adding hooks for these functions shouldn't be hard.

It can be done using proper matrix calculations, but it would be non-linear projection and would require really complex calculations. In every game it's simpler to just render to memory and then back on the screen; this also has the benefit of easy applying of post-effects.