I am trying to move an object on the 2D plane to follow the mouse position.

I converted the mouse's XY and depth buffer value to world space coordinates, but the problem is that when I run the program, the object is really far away. When I move the mouse, the object follows it, but also it is rotating on the Y and X axes.

What I want is for the object remain at its current Z coordinate and be able to move it up and down with mouse. Is my IDE incorrect or did I make a mistake in implementation?

I also know about this approach and I tried to implement it, but a few problems have occurred. I take mouse position X, Y and the depth buffer value and convert it to word space coords and then save those in the variables position and last_posiiton. Now the problem is that I need a plane that is parallel to the near and far plane. To check if mouse "position" and mouse and "last_position" intersect it I need its normal vector and it's origin. How do I get those two parameters?

Code I have so far:

#include <iostream>

#include <glad/glad.h>
#include <GLFW/glfw3.h>
#include <glm/glm.hpp>
#include <glm/gtc/matrix_transform.hpp>
#include <glm/gtc/type_ptr.hpp>
#include <glm/gtx/intersect.hpp>

#include "VBO.h"
#include "VAO.h"
#include "EBO.h"
#include "Camera.h"
#include "Shader.h"

#include "Cube.h"

void framebuffer_size_callback(GLFWwindow* window, int width, int height);
void mouse_callback(GLFWwindow* window, double xpos, double ypos);
void scroll_callback(GLFWwindow* window, double xoffset, double yoffset);
void processInput(GLFWwindow* window);

const unsigned int SCR_WIDTH = 800;
const unsigned int SCR_HEIGHT = 600;

float getObjectsDephtValue(float zComponent);

Camera camera(glm::vec3(0.0f, 0.0f, 3.0f));
float lastX = SCR_WIDTH / 2.0f;
float lastY = SCR_HEIGHT / 2.0f;
bool firstMouse = true;

float deltaTime = 0.0f;
float lastFrame = 0.0f;

int main()
    glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3);
    glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 3);

    GLFWwindow* window = glfwCreateWindow(800, 600, "Hello Window!", NULL, NULL);
    if (window == NULL)
        std::cout << "Failed to initialize GLFW" << std::endl;
        return -1;


    if (!gladLoadGLLoader((GLADloadproc)glfwGetProcAddress))
        std::cout << "Failed to initialize GLAD" << std::endl;
        return -1;

    glViewport(0, 0, 800, 600);
    glfwSetFramebufferSizeCallback(window, framebuffer_size_callback);
    glfwSetCursorPosCallback(window, mouse_callback);
    glfwSetScrollCallback(window, scroll_callback);

    //glfwSetInputMode(window, GLFW_CURSOR, GLFW_CURSOR_DISABLED);


    glm::mat4 projection = glm::mat4(1.0f);
    glm::mat4 view = glm::mat4(1.0f);
    glm::mat4 model = glm::mat4(1.0f);

    Cube cube1(1.0f, "D:/Textures for OpenGL/container.jpg", &view, &projection);
    Cube cube2(1.0f, "D:/Textures for OpenGL/grass.jpg", &view, &projection);

    glm::vec3 last_windows_coords(0.0f);

    while (!glfwWindowShouldClose(window))
        float currentFrame = glfwGetTime();
        deltaTime = currentFrame - lastFrame;
        lastFrame = currentFrame;

        glClearColor(0.2f, 0.3f, 0.3f, 1.0f);

        projection = glm::perspective(glm::radians(camera.Zoom), (float)SCR_WIDTH / (float)SCR_HEIGHT, 0.1f, 100.0f);
        view = camera.GetViewMatrix();

        double xpos, ypos;
        glfwGetCursorPos(window, &xpos, &ypos);
        ypos = 600.0f - ypos;

        GLfloat depth_comp;
        glReadPixels((int)xpos, (int)ypos, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &depth_comp);

        glm::vec4 viewport = glm::vec4(0.0f, 0.0f, 800.0f, 600.0f);
        glm::vec3 windows_coords = glm::vec3(xpos, ypos, depth_comp);
        glm::vec3 last_position = glm::unProject(last_windows_coords, view * model, projection, viewport);
        glm::vec3 position = glm::unProject(windows_coords, view * model, projection, viewport);


        cube2.setPosition(glm::vec3(0.0, 0.0, -3.0));

        last_windows_coords = windows_coords;



1 Answer 1


Since you are clearing the depth buffer at the beginning of each frame this call will always return the same value, so it's not clear to me why you are doing this:

glReadPixels((int)xpos, (int)ypos, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &depth_comp);

you are sampling an "empty" buffer.

To get the world position of a point that sits on a plane somewhere between the near and far plane you could do something like the following:

    // Projection of a point on a plane that sits halfway between near and far
    glm::vec4 myParallelPlaneProjection = (projection * glm::vec4(0, 0, -(near + (far - near)*0.5f), 1.0f));
    float myParallelPlaneDepth = myParallelPlaneProjection.z / myParallelPlaneProjection.w;
    glm::vec4 viewport = glm::vec4(0.0f, 0.0f, 800.0f, 600.0f);
    glm::vec3 windows_coords = glm::vec3(xpos, ypos, myParallelPlaneDepth);
    glm::vec3 position = glm::unProject(windows_coords, view * model, projection, viewport);

For the rotation issue I came up with this solution:

// Setting an appropriate model matrix for an object we want to keep "parallel" to the camera planes
cube1.Transform(glm::inverse(glm::lookAt(position, position + camera.Front, camera.Up)));

which is probably not ideal.

However, if this is the result you are looking for it should do the job (using a cone for a better reading of the scene):

enter image description here

  • \$\begingroup\$ But I don't need to draw a plane. I need to get normal vector an origin of this plane that is parallel to near plane and far plane and I don't know how to do it. \$\endgroup\$
    – tonn204
    Apr 2, 2022 at 21:52
  • \$\begingroup\$ This solution works almost perfect but I don't understand the math behind this. Can you explain it or give me a link to some resources which do that? Also, for instance when object is in position (0.0.0) and I click on to move it suddenly jumps further on the Z axis. Is there any solution to at least decrease this effect? \$\endgroup\$
    – tonn204
    Apr 3, 2022 at 19:46
  • \$\begingroup\$ You could read this great article about projection transformations. This one could be useful if you need an introduction to the topic. \$\endgroup\$ Oct 2, 2022 at 19:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .