# How to pick a 3D object with known mouse coordinates if the camera is rotated 60 degrees?

I am making a simple game engine for a simple RTS game and would like to be able to pick a cube and move it around with the mouse coordinates. The problem is that my camera is rotated 60 degrees just like any RTS game and I am not sure how to use gluUnProject..

• What were your problems using gluUnproject? Didn't you know what values to provide for some of the parameters or was it something else? – Roy T. Nov 14 '12 at 12:43
• Simply the 60 degrees tilting. I would like to get the correct z value to move objects – SPL Nov 14 '12 at 13:46
• How to pick objects in 3d is a topic of its own, but as far as gluUnProject goes it's not too difficult: just plug in the same matrices as you used when rendering! gluUnProject(winX, winY, winZ, modelViewMatrix, projectionMatrix, viewport, &objX, &objY, &objZ) will convert from window coordinates to world coordinates. Window coordinates should be normalized device coordinates (0..1 range, winZ is the value from the depth buffer). – ccxvii Nov 14 '12 at 15:59
• Look at picking tagged questions. – ErikEsTT Nov 15 '12 at 7:17
• Thank you. Picking is not good for this, or at least I didn't find anything that could give me not a face but a point on a face's surface. – SPL Nov 20 '12 at 16:33

Fundamentally, "what part of the world is under my cursor" is actually a problem of collision detection -- in terms of code, there's no difference between checking what's under the cursor, and checking what part of the world was hit by a fired bullet.

So, if we think of checking what's under our mouse cursor as 'firing a bullet', we're clearly firing it from the camera. And if you imagine that your screen is (for example) floating a meter in front of the camera, the bullet is being aimed to pass through a specific pixel on the virtual screen -- the pixel where the cursor is being drawn on the real screen.

If we know those two positions in the game world -- the camera position and the pixel position -- then finding out what's under that pixel is a trivial problem.

Now, we already know where the camera is in our game world, so that part isn't a problem. The only tricky part is figuring out where in our game world that virtual screen pixel is.

To get that game-world screen pixel position, I do this (pseudocode follows):

// first, get the cursor's window position, in pixels.
Vector2D pixelPos2D = WindowMousePosition();

// convert pixelPos2D from the range  [(0,0) .. (width,height)] to
// the range [(-1,-1) .. (1,1)]
pos2D.x = pos2D.x / (0.5f * WindowWidth()) - 1.0f;
pos2D.y = pos2D.y / (0.5f * WindowHeight()) - 1.0f;

// now work out how big our screen is, in world coordinates,
// if it's one meter in front of the camera.
float vertFOV = CameraVerticalFieldOfView();
float halfVertFOV = 0.5f * vertFOV;
float screenHalfHeight = tanf(halfVertFOV);
float screenHalfWidth = screenHalfHeight * WindowAspectRatio();

// screenHalfHeight and screenHalfWidth tell us (half) the actual in-game
// size of our screen, assuming that it's placed one meter in front of
// the camera.
//
// So relative to the camera, our pixel must be located at:
Vector3D pixelPos3D( pos2D.x * screenHalfWidth, pos2D.y * screenHalfHeight, 1.f );

// now, we need our pixel's position relative to the world, instead of
// relative to the camera, so we need to apply our camera's 3D
// transformation to it.
Vector3D pixelWorldPos3D = camera->GetLocalToWorldMatrix() * pixelPos3D;
// (or however your maths library performs matrix multiplication)


(Note that in this code, I assume that our camera looks forward along its positive Z axis, and that its X and Y axes match your window's X and Y axes. If your camera looks in some other direction, or has swapped its X or Y directions compared against your window's coordinate system, you'll need to make minor modifications)

And that's pretty much it. Now you know where your camera is, now you know where that virtual "pixel" is, and so now you can fire a pretend 'bullet' along that line using whatever method you already use for collision detection. It will tell you exactly what you hit, and exactly where.

There are other ways to do this as well. As noted in the question's comments, gluUnProject() is a common approach. Folks often will recommend reading back a pixel of the z-buffer (generally from the previous frame) to determine the z-buffer depth at the pixel to be queried, and then feeding the 2D pixel position plus that depth value into gluUnProject(), you can get where that point was in world-space. Down-sides of this approach are that at the time of calling gluUnProject(), you need to know the modelview matrix, the projection matrix, and the viewport settings. Plus, of course, needing to read back a pixel of the z-buffer, which almost certainly will cause an OpenGL pipeline stall. Plus, you need to be very careful in your code execution; it's easy for your z-buffer to be a frame or two old, which can result in serious inaccuracies when the camera or objects in the world are moving quickly.

Alternately, as Lars points out, if you do know those various matrices and viewport settings, then you can call gluUnProject() with an arbitrary non-zero 'winZ' value, and it will give you the world position of the specified pixel, at that distance forward from the camera. Using a winZ value of '1' should give the same final position that is calculated by my pseudocode, above, so you could use that as a less-code-required way to generate the direction for your virtual 'bullet' to fire, if you know the other information which it needs.

• (Actually, a more correct answer would probably be to put the screen on the near clip plane, and would be firing from the pixel's position on the near clip plane, firing directly away from the camera. But this answer was already too wordy/mathy, and this distinction won't make any difference for most games.) – Trevor Powell Dec 23 '14 at 6:34
• You appear to misunderstand what GLU's unproject does. It does a plain boring transform from window coordinates to world coordinates. Do it for two arbitrary WinZ values and you get two points you can form rays from, just like your mathy math math tries to do. The downside of GLU is that it has some dependencies on GL state like the viewport, and wants matrices in GL style. It in no way draws or uses the drawn geometry to do any read-back based picking. Please adjust your answer accordingly. – Lars Viklund Dec 23 '14 at 8:10
• Gah, good clarification, Lars! My brain was elsewhere. I've never actually used gluUnProject(), so I was going based on memory of overheard conversations. You're right; gluUnProject doesn't read back the Z-Buffer itself; that's usually the immediately-prior step in gluUnProject()-based answers, for determining the window 'depth' input value that gluUnProject() wants. Will amend my answer. – Trevor Powell Dec 23 '14 at 8:37