15
\$\begingroup\$

With XNA, I am displaying a simple rectangle which is projected onto the floor. The projector can be placed at an arbitrary position. Obviously, the projected rectangle gets distorted according to the projectors position and angle. A Kinect scans the floor looking for the four corners. Now my goal is to transform the original rectangle such that the projection is no longer distorted by basically pre-warping the rectangle.

My first approach was to do everything in 2D: First compute a perspective transformation (using OpenCV's warpPerspective()) from the scanned points to the internal rectangle's points und apply the inverse to the rectangle. This seemed to work but was too slow as it couldn't be rendered on the GPU.

The second approach was to do everything in 3D in order to use XNA's rendering features. First, I would display a plane, scan its corners with Kinect and map the received 3D-Points to the original plane. Theoretically, I could apply the inverse of the perspective transformation to the plane, as I did in the 2D-approach. However, since XNA works with a view and projection matrix, I can't just call a function such as warpPerspective() and get the desired result. I would need to compute the new parameters for the camera's view and projection matrix.

Question: Is it possible to compute these parameters and split them into two matrices (view and projection)? If not, is there another approach I could use?

Improve this question
\$\endgroup\$
3
  • 1
    \$\begingroup\$ XNA uses a View and a Projection matrix, but I think the end result = vector * view * projection. Why not try to make view an identity matrix and projection the inverse perspective matrix and see if that works? (Not 100% sure that this is exactly what happens) \$\endgroup\$
    – Roy T.
    Apr 14, 2012 at 20:41
  • 1
    \$\begingroup\$ How exactly did you compute a perspective transformation with warpPespective? I'm not familiar with OpenCV, but reading the doc it looks like this function just applies a perspective to an image. Or am I confused? Anyway, maybe adding more details on your first implementation would help. \$\endgroup\$
    – Laurent Couvidou
    Apr 24, 2012 at 22:31
  • \$\begingroup\$ You might want to take a look at the PCL library (pointclouds.org). Converting the depth image from the kinect gives you a point cloud with the camera at the origin, pointing along the z axis. You could then use ransac or another algorithm to search for the plane. \$\endgroup\$
    – Exilyth
    Jul 19, 2012 at 23:12

1 Answer 1

Reset to default
1
\$\begingroup\$

Since vector algebra is GPU friendly, normalizations and dot products can be used to find the four corners of the original plane as follow:

enter image description here

Given the projector point (P), the projected point (B), one arbitrary point on the plane that contains the distorted rectangle (Q), and the normal vector to that plane (n), the point of intersection (A) of the line from P to B, and the plane is given by

s = -dot_product(n, P - Q) / dot_product(n, normalized(B - P)) 
A = P + s * normalized(B-P)

Source http://geomalgorithms.com/a05-_intersect-1.html section Line-Plane Intersection

Improve this answer
\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .