How to check if a point is on the plane given bottom-right point, top-left point, normal and the quaternion rotation of the plane

Question

I know the world positions of A and B.

We are also given the vector normal and the quaternion rotation of the plane

How do I check if a point lies on the plane?

Answer 1

Let's assume your quaternion is set up so that it rotates the z+ vector to point along the plane normal, the y+ vector along the rectangle's "up" axis, and the x+ vector along the rectangle's "right" axis.

(This is what you'd get in Unity with Quaternion.LookRotation(planeNormal, Vector3.up), for example, assuming you want the rectangle's vertical axis to align as close to the world up direction as it can. That assumption, and the Vector3.up that encodes it, is the fourth piece of information we need, as I was decribing in our chat)

Then we can work this out very simply:

Vector3 ClosestPointOnPlane(Vector3 point, Quaternion orientation, 
                            Vector3 topLeft, Vector3 bottomRight) {

    Quaternion inverse = Quaternion.Inverse(orientation);

    Vector3 inRectangleSpace = inverse * (point - topLeft);


    Vector3 limit = inverse * (bottomRight - topLeft);


    inRectangleSpace.x = Mathf.Clamp(inRectangleSpace.x, 0, limit.x);
    inRectangleSpace.y = Mathf.Clamp(inRectangleSpace.y, limit.y, 0);
    inRectangleSpace.z = 0;

    return (orientation * inRectangleSpace) + topLeft;
}

Now you can compare the returned point to the one you gave it. If they're within the threshold you deem "close enough" you can count the point as being in the rectangle.

Because we don't have infinite-precision real numbers, many points on the rectangle won't be exactly representable, so doing it this way lets us count points that are not strictly "on" the rectangle, but as close as we can reasonably get with our number format and the rounding in our arithmetic.