Before I used GLU.gluUnproject to calculate my picking ray in my OpenGL game. Recently I switched to my own calculated matrices, and now I can forget the gluUnproject.

How can I calculate the picking ray with my own matrices?

My picking ray code (for the fixed pipeline):

public Ray getPickingRay() {
    FloatBuffer modelBuffer = BufferUtils.createFloatBuffer(16);
    FloatBuffer projBuffer = BufferUtils.createFloatBuffer(16);

    glGetFloat(GL_MODELVIEW_MATRIX, modelBuffer);
    glGetFloat(GL_PROJECTION_MATRIX, projBuffer);

    IntBuffer viewPort = BufferUtils.createIntBuffer(16);
    glGetInteger(GL_VIEWPORT, viewPort);

    FloatBuffer obj_pos = BufferUtils.createFloatBuffer(3);
    float middleX = (Display.getWidth() / 2)-0.01f;
    float middleY = (Display.getHeight() / 2)-0.01f;
    GLU.gluUnProject(middleX, middleY, 1.0f, modelBuffer, projBuffer, viewPort, obj_pos);

    Vector3f objPos = new Vector3f(obj_pos.get(0), obj_pos.get(1), obj_pos.get(2));

    Vector3f.sub(objPos, game.player.position, objPos);

    return new Ray(game.player.position, objPos);

At first I thought I just could use my own model- and projection matrix, but I have not found a way to manually calculate the GL_VIEWPORT intBuffer.

Thanks folks.


The values you'd get from glGet with GL_VIEWPORT (that you were previously passing to glUnProject) are the same values you have previously used to define the OpenGL viewport size using glViewport.

If, as noted in the referenced documentation, you've never called glViewport yourself, the width and height of the viewport are the dimensions of the (client area) of the window (and the X and Y values are both zero). You can therefore use the bounds of the client area of your window in the computation of your pick ray where glUnProject would have used the GL_VIEWPORT values.

  • \$\begingroup\$ So if I create an empty intbuffer and fill it with x, y, width, height in that order I have the GL_VIEWPORT? \$\endgroup\$
    – Basaa
    Jun 5 '13 at 15:27
  • \$\begingroup\$ So long as you have not modified the GL viewport with glViewport elsewhere, yes. \$\endgroup\$
    – user1430
    Jun 5 '13 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.