EDIT: I've removed some vagueness and replaced it with more of a description of the intended use case.

I'm looking for some suggestions for ways to infer what the player's "forward" vector is in a room-scale 360 experience. I'm not currently worried about extreme edge cases, especially if it sounds like the user may be intentionally not doing the intended thing, but I am interested in getting a decently accurate forward vector when they could be looking or facing any direction. I also don't care about "upness" or magnitude of the vector, so it'll end up being projected onto the (Unity's) X/Z plane and normalized.

I'm exploring the idea of punching as a means for locomotion. The player should move in the direction that they punch, which in itself I feel is an easy solve. The problem is, much like the problem of arm-swing movement, it is very hard and unnatural to punch exactly straight forward. People tend to punch at least a little across their body. This causes the player to weave back and forth through the world, like some sort of drunken master. I don't like the idea of using the headset's forward vector either. I want the player to be able to look around while playing; I feel it is an integral part of the experience to be able to look ahead one punch and plan the next move.

It's also important that sensor set up and room orientation should not be taken into account. I'm looking for something that could work in a large space with 360 tracking.

I've considered:

  • Grabbing an average position of the hands, getting the vector from the head to this new position. This is where I've spent most of my time, I think. I've tried a weighted average, where hand position gets more or less weight depending on distance from head. In practice, this ends up exacerbating the problem
  • Grabbing the vector from the head to the dominant hand
  • Dropping a raycast to the floor and using the hit geometry's forward vector
  • Getting a complicated IK rig setup, potentially hiding the geometry, and using the rig's hip placement/orientation to infer "forward"

None of these quite feel right to me, although the average-hand-position vector feels the right-est.

Is this a solved problem that I'm just not searching hard enough for? Anybody else have interesting solutions to this problem? Am I being too vague and should really just seek out a solution for a much more specific use-case and be happy with that?

  • \$\begingroup\$ Why do you need to know "forward"? Most of your examples seem like they're solved by tracking hands directly. \$\endgroup\$
    – THiebert
    Oct 13, 2018 at 0:44
  • \$\begingroup\$ Let's take the arm swing and speed bag locomotion examples for... example. It is very hard to move directly in a straight line if you are just tracking your hands in these situations. Or at least, its hard to move with any sort of precision. I end up slightly weaving side to side as I am continually correcting my path. \$\endgroup\$ Oct 13, 2018 at 2:09
  • \$\begingroup\$ There's almost a philosophical thing going on in this question: What exactly is "forward"? If I am rowing a row boat with my back in the direction of travel, am I going forward or backward? If I then quickly turn to look at the front of the boat instead and continue rowing, my "forward" changes, but my boat's stays the same. If I am playing paintball, I can run toward the enemy flag, and simultaneously shoot at an enemy to my left. Though simultaneous, both actions are directional toward their own different "forward" directions. \$\endgroup\$
    – Anko
    Oct 15, 2018 at 14:19
  • \$\begingroup\$ To get to the point: I think it would be best to edit this to focus on the specific problem you're trying to solve, and generalise only as necessary. \$\endgroup\$
    – Anko
    Oct 15, 2018 at 14:20
  • \$\begingroup\$ @Anko I've updated the question to remove some vagueness and add some details. \$\endgroup\$ Oct 15, 2018 at 18:02

1 Answer 1


You need a calibration step.

Exactly what you are measuring isn't that important (ie all of your suggested solutions should function) but you need to compare with the player's sense of direction. Its why (early) touch devices and current VR tech have it: the programmers can't be sure that their assumptions are true, but I'd they're close they can get the user to configure offset approximations (its also fun to mess with these calibrations, just don't mess with it so much you can't get back into them!).

Present a target to the player and all them to punch at it. Do this several times against targets in different places and you can get a feel for where the player expects "forward" to be (essentially by averaging the measured vectors together and computing an offset from mathematically precise). Eg. if you have them punch forward three times and left three times, you average both sets of 3 separately relative to the mathematical direction and can interpret any direction in between.

You'd want to take readings in three or four directions a bunch of times, then a roundup test against six or eight directions to check the results (eg left 3 times, right 3 times, forward 3 times, and up 3 times, then once each left, right, up, down, a little left and a little right) to check your computed offset against an unseen direction.

Oh and then round to the nearest 5 degrees during play. The player won't ever be 100% on target, but give a little leeway ("ehh, that was forwardish, we'll move the player mathematically forward") and they won't even notice (because the system works!). You can even autocorrect / aim assist to see if solid ground, goal, or similar object exists around (within a small distance of) the selected point. If so, move the player there instead. It allows some flexibility, as the player will aim in a general direction and the system will be able to intuit "yeah, that's what they meant."

  • \$\begingroup\$ I think this kind of calibration step provides for a pretty bad user experience. To that end, I was looking at a library that does gesture recognition using machine learning, and it seemed relatively useful. I'd be worried about the recognition speed at runtime though. The idea to round the vector to the nearest 5 degrees is interesting, though. I wonder if I could combine that with the idea of using the geometry's forward vector to get some decent results. \$\endgroup\$ Oct 19, 2018 at 17:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .