Setting: In our facility we have a 3D laboratory with 3 large displays (two edge-to-edge on orthogonal walls, one on the ground, also edge-to-edge). Its primary use-case is exploring 3D virtual environments. The control device is a Flystick, a 6 direction-of-freedom input device tracked by IR-cameras. One can query the stick's position in the room and its current orientation (rotation matrix).

Goal: I want to write a small tool which enables the user to use the stick as a pointing device to control the mouse cursor on the 3 screens.

Challenge: The stick does not have a pre-defined pointing direction. So even if I know its current rotation matrix I don't know which direction it is actually pointing at. That depends on its initial calibration (which may vary from application to application). So I need to figure out the stick's actual pointing direction on my own.

Plan: When starting my tool, I let the user point to the upper left corner of the first screen. Together with the current position of the stick I can calculate the pointing direction in the room's coordinate system. Now I transfere it into the stick's coordinate system by multiplying its inverse rotation matrix (at that moment) with that vector. This should give me the pointing direction in stick-space.

Now, if I want to determine the spot the user is pointing at for any other position & orientation, I need the stick's current position in room-space (check) and its pointing direction (also in room-space). To get that, I just need to apply the stick's current rotation matrix to the (stick-space) pointing vector which I determined during my calibration.

Results: So much for the plan. The calibration itself seem to work fine; and when only moving slightly from the screen corner the found pointing target seems to be reasonable. But when pointing for example on another screen, the results are totally wrong.

It seems that the stick's pointing direction in room space is incorrect.

Now the question is, is there a flaw in my approach? Did I miss something?

  • \$\begingroup\$ sounds pretty localized but neat so +1 \$\endgroup\$
    – GameDev-er
    Commented Jan 25, 2013 at 0:47
  • 1
    \$\begingroup\$ But it's specific to a single person in a single laboratory in the world, so I'm voting "too localised" for a Q&A resource site. If this question could be reformulated such that an answer would be useful to more than just one person in the world, that'd be ideal. (But if not, our chat or the various discussion boards would still be good places to ask!) \$\endgroup\$ Commented Jan 25, 2013 at 3:25
  • \$\begingroup\$ I agree that the setting is pretty specific, but I think the calibration of a 6 DOF tracking device is a general problem, for other 3D labs as well. GameDev is maybe not the right place, but I hope to find experts for alternative input devices here. \$\endgroup\$ Commented Jan 25, 2013 at 11:45

1 Answer 1


If you have the stick position and pointing direction, and if those are correct, one solution is to do a raycast.

Create a ray starting from your stick position (O) and expanding in its pointing direction (D), and test the intersections against your 3 planes. Check for instance this blog post for the math:

We can [...] represent the path the ray takes as a parametric equation in t:

P = O + Dt

We [can write] the plane equation in vector form as:

P . N = -d

The distance the ray has travelled when it hits the plane [is]:

t = (-d - O . N) / (D . N)

We can plug this value back into the ray equation to get the intersection point in 3D coordinates too.

The closest intersection (i.e. the smallest t) will determine which screen is touched, and your pointer position in room space.

You'll then have to transform that position from room space to this particular screen space. You can create a transformation matrix from screen space to room space, and use the inverse to transform your intersection point.

  • \$\begingroup\$ Thank you very much for your input. Unfortunately the intersection calculation is the part of my program that works very well (and similar to your approach). I am having problems determining the stick's pointing direction in room space. \$\endgroup\$ Commented Jan 25, 2013 at 11:48
  • \$\begingroup\$ @FrankSchlegel OK, I've edited your question to put an emphasis on that. Did you try to debug your stick pointing direction, are you absolutely sure that this is what is giving you trouble? \$\endgroup\$ Commented Jan 25, 2013 at 11:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .