1
\$\begingroup\$

I'm interested in creating a DDR (dance dance revolution) game using unity3D and Kinect. However I need to do this without actually using a pressure sensor pad, just a mat with grids and numbers on it.

Here is what the Grid is supposed to look like:

The Game Grid

I need to be able to track where the Mat is placed on the floor and a way to check which number is the user stepping on. The user feet coordinates can be tracked via skeletal tracking, however the grid coordinates and boundaries need to be extracted.

Is there a way to do this without using openCv and can opencv be used with Kinect and unity3d?

\$\endgroup\$
  • \$\begingroup\$ One cheapo way you could do it is to ask the user to calibrate it for you. At the beginning of the play session, ask them to stand on points 1 & 3. Give a few seconds' countdown or wait for a button input/gesture/voice command, then record the positions of the feet according to the skeleton tracking. Then repeat for points 7 & 9. Now you know the bounds and can infer the whole grid to within reasonable precision. Autodetection is of course nicer, but this manual calibration is fast to test, and could be worth keeping as an option in case autodetection fails. \$\endgroup\$ – DMGregory Feb 17 '15 at 3:03
1
\$\begingroup\$

Is there a way to do this without using openCv

Yes, but it will be difficult to do without some kind of image processing library.

can opencv be used with Kinect and unity3d

I believe you can use EmguCV, but don't quote me on that.

I need to be able to track where the Mat is placed on the floor and a way to check which number is the user stepping on.

You could go about doing 2D computer vision from first principles (extracting edges, using them to infer where the mat is, perhaps dividing it into cells, etc.); the method would be similar (if not identical) to extracting a checkerboard. But I would highly discourage this, because it will not be robust at all. It's also possible to do this using 3D computer vision (finding the floor plane, extracting points of a certain color to infer the mat region), but again I highly discourage this.

Instead of doing any of that, I recommend you attach some nice, big April Tags or ARTags to each of the cells in your board. These are patterns that are explicitly made very easy to detect using computer vision. You will probably be able to find a library for such tags that is compatible with C# and potentially even Unity. The tags will give you the position/orientation of each cell with respect to the camera. You can use these to infer world-space regions that you can try to detect the user's feet in.

\$\endgroup\$
  • \$\begingroup\$ Thank you for pointing me to EmguCV, i guess you're right it will need openCV or some sort of computer vision library \$\endgroup\$ – Sweta Dwivedi Feb 17 '15 at 4:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.