My vertex format has a float4 Flags element used to pass miscellaneous data to shaders. Each mouse-collidable object that is rendered has a unique ObjectID that is written to an ObjectID-rendertarget which can be CPU-sampled to index the hovered object. The remaining render targets are geometry/lighting data.

I've found this has advantages:

  1. I set (Flags.x = ObjectID) upon object creation so the vertex buffer only needs an initial upload and updates only when new ObjectID's (that will be rendered) are created. Everything else is per-object/instance. Essentially, "no" overhead added by using a Flags component for ObjectID.
  2. I'm semi-certain that, since I have to render everything anyway, outputting one additional color to one additional rendertarget is already cheaper than ANY clever bounding/container collision testing algorithm. Currently, I copy the entire ObjectID-rendertarget back to the CPU, but even that could be further optimized.
  3. The depth buffer (that I'm using anyway) automatically ensures that the pixel I sample is the ObjectID of the top-most, visible, mouse-collidable, object. I don't have to check if the mouse position/ray is within the bounds of anything, intersects with anything, visibilities, nothing. All of that is already sorted out by rendering the scene properly.
  4. Rendering the interface components first and reusing the ObjectID depth-buffer to render the world, reduces the fill-volume for the 3D scene; drastically, if a lot of interface is visible.

I've found only one disadvantage:

  1. Since I render the interface first, if the camera has moved (the world hasn't been rendered yet), the WorldCoordinate available during Update() is incorrect. If the camera hasn't changed, the coordinate is correct. Immediately after rendering to the geometry buffers, but before any cursor-dependent shaders are invoked, I update the CPU-buffer and WorldCoordinate. The CPU-buffer is only updated when necessary and, at most, once per-frame. The "second sample" is during the following Update() and just re-reads the (wrong if the camera has moved) value from the byte array. Still, no significant overhead and this is basically just a slight annoyance because I'm just debugging the value anyway. Currently, when the WorldCoordinate is usefully-used (during render), it is correct.

Overall, I've got pixel-perfect, anything-on-the-screen, mouse-collision detection, with zero* additional overhead beyond standard rendering.
*Minimal and/or approximately-fixed, regardless of screen/scene complexity

Why is this method not popular and/or tutorialized?
-I assume most new-ish games post-process geometry buffers but I found nothing similar to this and just worked my way through the issues as they came up. To me, it appears as though I've got collision detection working for all of the objects/classes that I haven't even written yet. All of the "tech" involved here is obvious and easy, so I feel like I must be missing something; a future problem that I can't think of yet.

-Comments/critiques of the "algorithm" are welcome. Best answers would point out critical flaw(s) and, for bonus points, work-arounds.

-If this has been done before, is there an accepted best-practice? Links to similars would be ok.

  • \$\begingroup\$ This is a very old technique, roughly called the "selection buffer" technique. It has been used as early as OpenGL 1.1. It is definitely the standard technique used for mouse selection in games. Early implementations of the technique relied on hacky selection buffers that were only supported by certain vendors, but as soon as modern shaders came about it has been used extensively. \$\endgroup\$
    – mklingen
    Mar 19, 2015 at 13:57

1 Answer 1


Well, it's not new and it's a proper method. I think a common term for this "color picking", at least that's what i call it.

As long as you are fine with picking only one object at a time, and that object being the one on top, there are no critical shortcomings. Even multi-select clicks are possible to a certain extend by mixing/adding colors/ids.

However, consider how often a user actually clicks versus the amount of processing. You will in one way or another have to write a full screen of extra values each frame, even if the user isn't clicking at all. How many clickable objects do you actually have to consider when the user is clicking? Even a screen cluttered full of symbols might be faster to traverse when a click is happening then the accumulated time/power spend on copying something each frame you probably won't use even every 50th frame.

  • \$\begingroup\$ You can pick multiple objects by scanning a bounding box. Also you only should render the selection buffer when the user clicks. You just register a click event, and when that happens the next frame will render a selection buffer. \$\endgroup\$
    – mklingen
    Mar 19, 2015 at 14:10
  • \$\begingroup\$ With multiple selection i meant objects occluding each other, not objects visible side by side. Those are obviously easily selected. \$\endgroup\$
    – Jonas Beck
    Mar 19, 2015 at 14:12
  • \$\begingroup\$ Oh I see. That's tricky, but I wonder under what circumstances that would be useful to do. \$\endgroup\$
    – mklingen
    Mar 19, 2015 at 14:14
  • \$\begingroup\$ And i agree that you should do the selection-buffer-rendering only when a click is happening. Still some overhead remains as you won't create/destroy the needed buffers only for a click. That being said, i totally like method - did not want to put it down if it sounded like that. \$\endgroup\$
    – Jonas Beck
    Mar 19, 2015 at 14:17
  • \$\begingroup\$ Fantastic; thank you both! This is technically the "hover"-buffer so I need to update it while not clicking. That being said, you've given me some good ideas for further optimizing this. \$\endgroup\$
    – Jon
    Mar 20, 2015 at 4:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .