What method replaces GL_SELECT for box selection?

Last 2 weeks I started working on a box selection that selects shapes using GL_SELECT and I just got it working finally.

When looking up resources online, there is a significant number of posts that say GL_SELECT is deprecated in OpenGL 3.0, but there is no mention of what had replace that function.

I learnt OpenGL 1.2 in back in college 2 years back but checking wikipedia now, I realise we already have OpenGL 4.0 but I am unaware of what I need to do to keep myself up to date.

So, in the meantime, what would be the latest preferred method for box selection?

EDIT: I found http://www.khronos.org/files/opengl-quick-reference-card.pdf on page 5 this card still lists glRenderMode(GL_SELECT) as part of the OpenGL 3.2 reference.

• It's unfortunate how outdated a lot of the college courses are. I was taught all immediate mode rendering when I took a class a few years ago. – MichaelHouse Dec 12 '12 at 17:55
• @Byte56 with immediate rendering you can get stuff done fairly quickly - in modern opengl there's just tons of stuff you need to know before you get a pixel on screen. – Jari Komppa Jan 8 '13 at 9:02

The way I do picking in core OpenGL is to assign a unique color to every object in the world and draw only the vertices (no texcoords, normals, etc.) with the assigned color to a FBO.

Once all the objects have been drawn, you can call glReadPixels at the mouse coordinate with a width and height of 1 to get the pixel color at that point. Then all you have to do is lookup which object was assigned to that color and return it.

If you want to drag-select a rectangle, do the same thing, but store the pressed mouse coordinate and change the parameters of glReadPixels to read back the selected rectangle between the pressed mouse coordinate and the released mouse coordinate.

• I use glReadPixel for a point selection. But for a large area, there will be lots of pixels to enumerate? More than necessary? – Jake Dec 12 '12 at 18:11
• It shouldn't be that slow, but if there's a noticeable performance issue with that method, you could try creating the FBO at half the screen size effectively reducing the amount of enumerating you need to do by 3/4. – Robert Rouhani Dec 12 '12 at 18:18
• You can also look into occlusion queries, I haven't used them myself but I've read that they can be used for something like this. – Robert Rouhani Dec 12 '12 at 18:19

In modern OpenGL you shouldn't use GL_SELECT for objects picking since OpenGL is an API for hardware accelarated graphics drawing. But there are other methods which you can use.

As @Robert wrote, you can use color selection in order to pick objects on the screen. Another thing you can do is to cast a ray from the mouse cursor to the game world and check for collisions between the ray and the world's geometry.

To construct the ray, you will need to get the mouse cursor's position in world space. That position will be of use to you in order to calculate the ray's direction. The ray's origin is the position of your camera. Now that you have the ray, you can make an intersection test between the ray and the geometry you are trying to pick. If the test succeeds, then you have a clicked object, else you don't have one.

The code concerning the ray's construction is the following:

Ray Mouse::GetPickRay(mat4 projMat, mat4 modelMat,
vec4 cameraPos,
int windowWidth, int windowHeight)
{
float mouseX = float(this->currentPosition.x);
float mouseY = float(this->currentPosition.y);

vec4 mousePos_clipSpace = vec4(
((mouseX * 2.0f) / windowWidth) - 1.0f,
(1.0f - (2.0f * mouseY) / windowHeight),
0.0f,
1.0f);

vec4 mousePos_viewSpace = inverse(projMat) * mousePos_clipSpace;

mousePos_viewSpace = mousePos_viewSpace / mousePos_viewSpace.w;

vec4 mousePos_worldSpace = inverse(modelMat) * mousePos_viewSpace;

vec4 rayDirection = normalize(mousePos_worldSpace - cameraPos);
vec4 rayOrigin = cameraPos;

return Ray(rayOrigin, rayDirection);
}


This article describes very well the logic behind the unprojection of the mouse cursor's coordinates and the creation of the ray.

The only thing that's left is to check for intersections between the ray and the world's geometry. You can find various algorithms on the Internet about how to do that.

• Just to confirm, are you suggesting this for box slection? – Jake Dec 14 '12 at 3:56
• Well, it can do well for any kind of selection so yes. The only thing you have to do is to find a good ray-box intersection algorithm. – Tsvetan Dec 14 '12 at 4:40

You could use something like Bullet.

It's a physics library but you can just use it for ray collision. I (should) be using it soon for my project.