I've recently got into 3D and I came up with a few different "tricky" rendering techniques. Unfortunately I don't have the time to work on this myself, but I'd like to know if these are known methods and if they can be used in practice.

Hybrid rendering

Now I know that ray-tracing is still not fast enough for real-time rendering, at least on home computers. I also know that hybrid rendering (a combination of rasterization and ray-tracing) is a well known theory. However I had the following idea: one could separate a scene into "important" and "not important" objects. First you render the "not important" objects using traditional rasterization. In this pass you also render the "important" objects using a special shader that simply marks these parts on the image using a special color, or some stencil/depth buffer trickery. Then in the second pass you read back the results of the first pass and start ray tracing, but only from the pixels that were marked by the "important" object's shader. This would allow you to only ray-trace exactly what you need to. Could this be fast enough for real-time effects?

Rendered physics

I'm specifically talking about bullet physics - intersection of a very small object (point/bullet) that travels across a straight line with other, relatively slow-moving, fairly constant objects. More specifically: hit detection. My idea is that you could render the scene from the point of view of the gun (or the bullet). Every object in the scene would draw a different color. You only need to render a 1x1 pixel window - the center of the screen (again, from the gun's point of view). Then you simply check that central pixel and the color tells you what you hit. This is pixel-perfect hit detection based on the graphical representation of objects, which is not common in games. Afaik traditional OpenGL "picking" is a similar method.

This could be extended in a few ways:

  • For larger (non-bullet) objects you render a larger portion of the screen.
  • If you put a special-colored plane in the middle of the scene (exactly where the bullet will be after the current frame) you get a method that works as the traditional slow-moving iterative physics test as well.
  • You could simulate objects that the bullet can pass through (with decreased velocity) using alpha blending or some similar trick.

So are these techniques in use anywhere, and/or are they practical at all?

  • \$\begingroup\$ I think this is a very interesting question, +1! \$\endgroup\$
    – ashes999
    Mar 27 '12 at 13:10
  • 2
    \$\begingroup\$ What's the question though? "What we think" about these approaches isn't suitable as answer, because this isn't a discussion forum. Also the two special renderer use-cases are quite different.. while the latter might be used, I don't think the first one is... \$\endgroup\$
    – bummzack
    Mar 27 '12 at 13:15
  • \$\begingroup\$ @bummzack My main question is if these are used in any product, or if they could be fast enough (even in theory) to be used in the future instead of the traditional methods. \$\endgroup\$
    – maul
    Mar 27 '12 at 13:22
  • \$\begingroup\$ While interesting, this "question" seems like it's not a question for this site. "Are these methods currently used?" - Why does it matter? "Are they theoretically fast enough to be used in the future?" - Probably, but again, why does that matter? Seems like a good discussion question, but not a fit for this site. \$\endgroup\$
    – House
    Mar 27 '12 at 15:00
  • \$\begingroup\$ the new currently-in-works id tech engine is said to be a combination of rasterization and ray-tracing \$\endgroup\$
    – dreta
    Mar 27 '12 at 15:03

As far as I know, neither technique you described is used as presented in any modern game. The first technique has, if nothing else, some serious image quality concerns and the second (while similar to selection buffer picking) adds nothing to the table and suffers all the drawbacks of selection buffer picking.

It is difficult to say from these theoretical perspectives what the performance of these techniques would be and how they'd compare to alternatives. Much of it will depend on the complexity of the underlying scenes, the size of the desired render targets (as well as their bit depth), what other optimization can be made for a particular implementation, et cetera. I would be confident in saying that they'd probably be on-par with existing techniques -- certainly they won't be so much faster as to overwhelm their disadvantages. You're effectively trying to turn everything into an image processing problem and image processing is not all that fast, and generally has accuracy complexities.

I'll cover some of the major disadvantages for each proposal below:

Hybrid rendering - what you are essentially describing here is a technique to reduce the cost of ray-tracing by reducing the amount of pixels you actually have to trace rays for. This is not dissimilar from simply reducing the resolution of the render surface, which can allow for ray-tracing computations to be done in real-time. So the technique is certainly feasible from that perspective.

However, it has some pretty serious drawbacks. First, you'd incur the readback cost for the render target to get it back on to the CPU (faster these days, but still less than ideal) and you'd have to iterate the entire render surface to look for pixels of the "special" colors anyway, and then perform a reverse transformation (so you'd probably want to readback the depth buffer as well) to find out where on the source "interesting" object the corresponding pixel was before you can start ray-tracing.

You could eliminate the readbacks by simply not rendering the important objects on the GPU, processing them only on the CPU using your raytracer into a separate buffer which is pre-filled with a transparency color. Then you can render that buffer over the GPU's framebuffer using an appropriate blend mode. While this would alleviate the readback issues, you'd still have what I think is the second (and larger) problem:

It probably won't look very good. You'd have a pretty big disparity in the visual fidelity of your important/non-important objects, plus you'd have lost most of the information you'd need to get some of the really cool things ray-tracing gives you, like local reflection for those non-important objects. Consequently this technique would probably only be useful for a highly stylized visual direction, and honestly there are probably better ways to achieve such a direction -- assuming it would even look good at all.

Rendered physics - this is very similar to selection-buffer style picking, as has been noted. It also reminds me of hierarchical occlusion maps. The main difference is that you're concerning yourself with the rendering of only a single pixel (or very small set of pixels). You'd probably be best off doing this on the CPU, because the bandwidth overhead to set up and ship things to the GPU would probably be wasteful. It, again, is feasible, but has some drawbacks.

It's consumes a fair amount of resources for performing a single task -- detecting a "hit" -- that is useful only, really, to that task. Traditional methods of hit detection using geometrical computations rely on data that already exists for physics computations and are themselves reusable (for drawing tracers, or determining where and how sounds might play, et cetera). You're not really gaining anything by picking in this fashion.

The technique is pixel-perfect, sure, but that's only a real desirable advantage in 2D games. In 3D games, a pixel ends up being a relatively poor resolution unit of measure as objects get further from the camera. Also, once you move beyond a 1x1 pixel render surface you start having to include depth information and do math on the pixels in the render target (two colors are present, which object was closer? which object consumes more of the hitscan surface area? et cetera).

  • \$\begingroup\$ Thank you, I was hoping for an answer with detailed explanation like this. \$\endgroup\$
    – maul
    Mar 27 '12 at 15:35
  • \$\begingroup\$ Great answer! And congrats on just hitting 20k reputation ;-) \$\endgroup\$ Mar 27 '12 at 16:49

Rendered physics

I believe this is a form of 'picking', and is commonly done.

enter image description here

  • \$\begingroup\$ Can you give any detalied measurements of speed and accuracy against the traditional physics method? Or maybe the name of a game/engine that does this? \$\endgroup\$
    – maul
    Mar 27 '12 at 13:59
  • \$\begingroup\$ Also, just to clarify, the image you pasted is not exactly what I had in mind. My idea would have the proper spaces between the leaves on the right side. \$\endgroup\$
    – maul
    Mar 27 '12 at 14:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .