# Calculate object coordinates from window coordinates using inverse projection matrix

What I am trying to to is placing 3D Objects (A Sphere) in the world when I click there.

I am using a perspective projection so the first step after calculating the canonical clip coordinates from window coordinates is to reverse the perspective transformation:

``````    void addSphere(int screenX, int screenY) {
// invert y axis
screenY = screenHeight - screenY;

// determine depth of the pixel
GLfloat z;
glReadPixels(screenX, screenY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &z);

// transform x and y to canonical clipping space
GLfloat x = -1 + (screenX / (float)screenWidt) * 2;
GLfloat y = -1 + (screenY / (float)screenHeight) * 2;
std::cout << "<" << x << "," << y << "," << z << ">" << std::endl;

// now we reverse the projection which means we have object coordinates
// given the MODELVIEW Matrix is identity
glm::vec4 center = glm::vec4(x, y, z, 1);
center = inverseProjection * center;
std::cout << glm::to_string(center) << std::endl;

// add the sphere centered at the calculated position
Sphere *sp = new Sphere(center);
sphereList->push_front(sp);
}
``````

The big question is what do I insert for the forth value for the center vector? As far as I see there is no method to get the transformed w value from the framebuffer.

-
not entirely sure, but I think what you need is the "unproject" function. It transforms points in the 2d space into the world. glm.g-truc.net/api-0.9.4/… – Jari Komppa Jun 3 '13 at 12:13
This is not production code. I try to understand what is going on there. But thanks for pointing this function. I maybe use it after understanding what is going on. – Alexander Theißen Jun 4 '13 at 14:01

As it is normalized clip coordinates (x/w, y/w, z/w, 1) that are finally mapped to the viewport, 1 is the correct w value.

Your z coordinate however is wrong, because while in clip coordinates, z is in [-1,1], this range is then linearly mapped to the range set by glDepthRange (usually [0,1]).

Also after inverse projecting, you probably need to homogenize your coordinate, i.e. divide by the resulting w value.

-
Okay I see the thing with z. You are absolutely right. I forgot the glDepthRange. I also forgot about the w-devision. Which leads to another question: The values in the frame buffer from which glReadPixels reads are w-devided which means z should be constant (-1 I think after inspecting the projection matrix, because w is -z). But obviously this is not the case (z value is at least needed for depth testing). Is z as, an exception, not w-devided? – Alexander Theißen Jun 4 '13 at 13:58
Now you are confusing things. w_clip is usually proportional to z_eye, but z_clip != w_clip (otherwise the whole z-buffer approach would be trivial). – Daniel Flassig Jun 4 '13 at 17:05
Note also that I completely agree with @Sean about ray casting in general. Depends how much precision/performance you need with respect to the work you may invest. – Daniel Flassig Jun 4 '13 at 17:09
You are right. I mistakenly thought z_eye = z_clip. Now it makes sense. You said I have to homogenize AFTER inverse projection. To be strict I have to un-homogenize before inverse projection (multiply with w), right? How did you know your method is correct? I thought about this because it wasn't obvious for me. I came up with a proof that yours is exactly the same as un-homogenize before inverse projection and now I can accept your answer with the good feeling of having understood it all. The raycasting approach may be the better one for most cases but this one fits my question better. – Alexander Theißen Jun 4 '13 at 18:06

You should not read a depth value for this.

You need to use both the inverse projection matrix (to transform to view space) then the inverse camera matrix (to transform to world space). Transform your NDC point (x,y,0,1) to view space using the inverse projection matrix. Set the depth to your near plane depth, remembering that typically OpenGL apps uses a negative Z axis for forward view vector, so you probably want `-nearZ`. Then transform from view to world space using the inverse camera matrix. Your point is now in world space on the camera's near plane.

This should be enough to do a ray cast (from the camera's position through the transformed point on its near plane) to find a ground plane to create your objects on, or you can normalize and scale this vector to put the sphere at some specific distance from the camera.

-
Is there any other reason besides performance (glReadPixels() stalls the pipeline, right?) to use raycasting instead of reading the z-Value from framebuffer? Is it because we want to click "through" objects occluding the "ground"? Can you please have a look at the comment I made under @DanielFlassig comment? How to you deal with w-Division in your method? Did you just assume that I know that I have to homogenize the eye space coordinates? – Alexander Theißen Jun 4 '13 at 14:27
Note that raycasting from the camera's origin to the world point on the near plane will only work for typical perspective matrices. It will give the wrong result for orthogonal matrices and more exotic projection matrices where the ray that corresponds to a single 2D location may not actually pass through the camera's origin. I think a more generic solution would be to inverse-project to find the world points of the screen location on both the near and far planes, then raycast between them to find the target world point. – jcl Apr 27 at 5:11