2D World Mouse Coordinates - Matrices

Question

I'm trying to get the world coordinates out of a mouse click. Currently I'm doing the following:

var invertedMatrix = mat4.create();
var viewMatrix = activeCamera.getMatrix();
mat4.invert(invertedMatrix, viewMatrix);

var worldCoord = Vector2.transformMat4(new Vector2(mouseX, mouseY), invertedMatrix);

But the values I'm getting are away off. The Y for instance, is between 0 (top of the screen) and 200.000+ on the bottom of the screen. Also, even if I move the camera, let's say 50 in the x axis, if I click in the same spot before and after, the value calculated is the same.

I'm using gl-matrix to generate and calculate the matrices. The used active camera transform was created by using the following instructions:

 mat4.ortho(this._matrix, -this.viewWidth / 2.0, this.viewWidth / 2.0, this.viewHeight / 2.0, -this.viewHeight / 2.0, 0.0, 1.0);
 mat4.translate(this._matrix, this._matrix, [-this.x, -this.y, 0]);

I've checked both the vector2 transform and matrix invert for any issues but there seems to be none which makes me believe it can be the ortho projection itself but I'm currently lacking the knowledge to understand the problem.

The transform calculation:

    return new Vector2(
       (mat[0] * vec2.x) + (mat[4] * vec2.y) + mat[12],
       (mat[1] * vec2.x) + (mat[5] * vec2.y) + mat[13]);

Any ideas? thanks!

Answer 1

Ok mystery solved.

The issue was because I wasn't normalizing the mouse x and y before applying the calculations. After doing the following everything started to work correctly:

var x = (2.0 * mouseX) / screenWidth - 1.0;
var y = 1.0 - (2.0 * mouseY) / screenHeight;