# Selecting the closest point in perspective view mode

I'm trying to improve my algorithm for selecting points in perspective view mode (OpenGL/Qt/C++). The current implementation works as follows.

1. The user clicks on a certain (x,y) position (in Window space)
2. Traversing the pipeline from Window space back to World space (that is, Window → NDC → Clip → Eye → World), this defines a point a on the near clipping plane and a point b on the far clipping plane
3. The distances from the line between a and b and the vertices (available in World space) are determined (see e.g. this article on Wikipedia)
4. The point with the minimum distance to the line is selected

Now, this works fine when using orthogonal view mode. However, when visualizing something in perspective view mode, there are some cases where the result might be a bit off.

As example, let's consider the wireframe of a cube. In the illustration below, the point (x,y) is the point where all white lines meet. These lines are orthogonal to the line from a to b which is not visible in this orientation — the second illustration shows a rotated view.

The point that eventually gets selected is highlighted in green. However, as the blue point seems to be closer to (x,y) in perspective mode, the user probably expects this point to get selected instead. Somehow the computed distances have to be scaled in order to take this effect into account. How is this usually done?

• Sounds like rather than computing the 3D distance of each point from the unprojected line, you want to compute the 2D distance of each point's pixel position from the clicked point on the screen. Commented Jan 10, 2016 at 15:51

The step you are missing is the perspective divide. To calculate the value to divide by you have to do the perspective matrix math anyway so you might as well convert the point from world space into screen space and solve the problem in 2D screen coordinates.

If you're curious about the math involved you can read up on how the perspective matrix works here

• Of course, that makes sense. However, as I have (a lot of) points in World Space coordinates, I was wondering how to scale the distances without having to project everything to Window Space. Commented Jan 12, 2016 at 18:42