Project 3D points to a plane, then project their bounding box back to 3D

Question

I have a table in 3D space, represented by a plane.

I want to project arbitary points representing an object (cup, toy, etc) onto that table, run 2D principal components analysis to get an oriented bounding box (https://www.sciencedirect.com/topics/computer-science/oriented-bounding-box)

This gives me 2D oriented bounding boxes (defined by 8 corner points) that lie along the table plane. Now, I want to project these points back to 3D somehow so I can get 3D bounding boxes that are aligned with the plane.

The projection from 3D to 2D plane is not invertible.. so how do I do this?

Answer 1

As you go through your points, before you project them to the plane, look at their height off the plane. Note down the greatest and least height off plane that you encounter for each object.

Once you have the four corners of your planar bounding box, you can duplicate them to four at your minimum height, and four at your maximum height, to get all 8 corners of a 3D bounding box.

This box is guaranteed to contain your object. If a point were outside it horizontally, you would have chosen a larger planar bounding box to encompass it. If a point were outside it vertically, you would have recorded a higher max height or a smaller min height.