I've been stuck for a month trying to get gluUnProject working. After my attempts to use gluUnProject failed (as well as attempts to implement gluUnProject functionality manually) I implemented method to get ray using angles.

 float RadToDeg = 180.0f / (float)M_PI; // conversion coefficient to degree
 float DegToRad = 1 / RadToDeg; // conversion coefficient to radians 

 float DeltaX = tan(fovy/2 * DegToRad) * 2 * (Width / 2 - winX) * zNear / Height;
 float DeltaY = tan(fovy/2 * DegToRad) * 2 * (Height / 2 - winY) * zNear / Height;
 float AngleX = atan(DeltaX / zNear) * RadToDeg;
 float AngleY = atan(DeltaY / zNear) * RadToDeg;

 float DeltaXY = sqrt(DeltaX*DeltaX + DeltaY*DeltaY);
 float AngleXY = atan(DeltaXY / zNear) * RadToDeg;
 float AxisAngle = atan(DeltaX / DeltaY) * RadToDeg;
 vec3 RotationAxis = rotate(Camera->X, AxisAngle, Camera->Z);
 if (DeltaY < 0) { RotationAxis = -RotationAxis; }

 vec3 ray = rotate(-(Camera->Z), AngleXY, RotationAxis); //getting ray direction
 ray = vec3(ray.x * scale, ray.y * scale, ray.z * scale); //scale ray if needed

 vec3 RayEndPoint = Camera->Position + ray;

The main idea behind my method is to rotate ray pointing perpendicularly from the middle of the screen (like in first person shooters) to position where you clicked.

Necessary values are fovy angle (field of view in vertical direction), zNear (distance from Camera to near plane) and Camera Transformation Matrix(Camera->X, Camera->Y, Camera->Z, Camera->Position - 1st, 2nd, 3rd and 4th columns). Difference between Camera Transformation Matrix and View Matrix here.

Besides we need some method rotate(vec3 vector,float angle,vec3 axis) that will rotate any vector at any angle around any arbitrary axis. Explanation how it works will consume too many lines (check photo of my calculations).calculations

Q: Is any easier or better way to do it? Any information or links welcomed.

  • \$\begingroup\$ Yes, there's an easier way: fix your bugs with the matrix version. That's the easiest way to do it, by far. I promise. :) \$\endgroup\$ Commented Sep 7, 2015 at 18:44

2 Answers 2


Just multiply a clip-space coordinate (x and y are screen-coords ranging from [-1;1], z=0, w=1) with the inverse of the view-projection matrix. Then you need to divide the resulting homogenous vector with its w-component. This gives you a position on the desired ray. To get the ray direction just subtract the camera position and normalize the result.

While this is surely not the best solution in number of arithmetical operations, I find it super intuitive. Since it relies on function which are included in most 3D-math libraries, it is usually also super short in its implementation.

  • \$\begingroup\$ I just realized that this is roughly what gluUnProject is supposed to do. opengl.org/sdk/docs/man2/xhtml/gluUnProject.xml \$\endgroup\$
    – Wumpf
    Commented Sep 6, 2015 at 20:14
  • \$\begingroup\$ I tried that already see my other question with no success. I undestand that it should work - so many topics and examples, but I'm missing something. \$\endgroup\$
    – mioe
    Commented Sep 6, 2015 at 20:16

A month ago I came across a youtube video which explains how to do it, although this video is a tutorial for Java, you will probably be able to use it.

The video was based on this text article.

I hope this is usefull for you.

PS: This tutorial uses the LWJGL Matrix functions, this StackExchange answer has some code which you will be able to copy if you need the inverse of a matrix.

  • \$\begingroup\$ I've already implemented such approach and explained here. What I was asking in current thread is how to get ray using angle deviations from other ray, e.g. get ray 30 degrees left and 45 degrees down from some ray we have. \$\endgroup\$
    – mioe
    Commented Sep 7, 2015 at 18:53
  • \$\begingroup\$ @mioe So you are asking for a 3D vector multiplication? \$\endgroup\$
    – jbakker
    Commented Sep 7, 2015 at 19:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .