Raycasting mouse coordinates to rotated object?

Question

I am trying to cast a ray from my mouse to a plane at a specified position with a known width and length and height.

I know that you can use the NDC (Normalized Device Coordinates) to cast ray but I don't know how can I detect if the ray actually hit the plane and when it did. The plane is translated -100 on the Y and rotated 60 on the X then translated again -100.

Can anyone please give me a good tutorial on this? For a complete noob! I am almost new to matrix and vector transformations.

Answer 1

There's a list of different collision types (ray-plane included) that can be found here. One of the better sources of ray-plane intersection can be found here.

With a c++ implementation:

// intersect3D_SegmentPlane(): intersect a segment and a plane
//    Input:  S = a segment, and Pn = a plane = {Point V0;  Vector n;}
//    Output: *I0 = the intersect point (when it exists)
//    Return: 0 = disjoint (no intersection)
//            1 =  intersection in the unique point *I0
//            2 = the  segment lies in the plane
int
intersect3D_SegmentPlane( Segment S, Plane Pn, Point* I )
{
    Vector    u = S.P1 - S.P0;
    Vector    w = S.P0 - Pn.V0;

    float     D = dot(Pn.n, u);
    float     N = -dot(Pn.n, w);

    if (fabs(D) < SMALL_NUM) {           // segment is parallel to plane
        if (N == 0)                      // segment lies in plane
            return 2;
        else
            return 0;                    // no intersection
    }
    // they are not parallel
    // compute intersect param
    float sI = N / D;
    if (sI < 0 || sI > 1)
        return 0;                        // no intersection

    *I = S.P0 + sI * u;                  // compute segment intersect point
    return 1;
}

Answer 2

The are two processes involved here. The first one is to obtain a ray (or line) toward the place the user feels the mouse is pointing to. The second one is to resolve the line and plane intersection equation.

Use the camera view-projection matrix to resolve the direction of the mouse. MVP matrix transform a point in space into screen coordinates so one can think that the inverse would transform a point in screen space into world space. As the camera is projecting the scene into a virtual plane called near plane you will get a point within that plane. Then trace a line from the camera position towards that near-plane point and you obtain a ray that goes into the scene. An example in java:

public Vector3 rayDirectionForViewportCoordinate(float x, float y){
    float normalized_x = 2.f * x / _viewportWidth - 1.f;
    float normalized_y = 1 - 2.f * y / _viewportHeight;


    Matrix.multiplyMM(mvp_f, 0, this.getProjectionMatrix().getAsArray(), 0, this.getViewMatrix().getAsArray(), 0);  
    Matrix.invertM(inv_mvp_f, 0, mvp_f, 0);

    Matrix4 inv_mvp = new Matrix4(inv_mvp_f);
    Vector4 near_point = inv_mvp.mul(new Vector4(normalized_x, normalized_y, 0 , 1));

    float w = 1.f/near_point.getW();
    Vector3 point = new Vector3(near_point.getX()*w, near_point.getY()*w, near_point.getZ()*w);
    point.subNoCopy(_position);
    point.normalizeNoCopy();
    return point;
}

After you have obtained the ray, use any plane-line solver out there to test at which place that casted ray intersects your plane.

Move the ray origin and direction into object space (your rotation&translation matrix you want to check against). Use the inverse of the model matrix for that specific object. For example by multiplying the ray's origin against the inverse of the object model matrix you effectively move the origin into object space. Do the same with the direction and check if the line hits the plane using. Alternatively, rotate the plane and do the maths in world space.

If this is not clear enough drop me a message.