How to remove jitter from motion input?

Question

I am writing a Minecraft mod that supports input from the Razer Hydra. It is a pair of motion controllers (one for each hand) that provide incredibly accurate position and rotation information.

For the purpose of this question, rotating the right controller on the Y-axis cause the player's character to look left or right (yaw), and rotating on the X-axis makes the player look up and down (pitch).

The input from the controller is mapped directly to the character's heading. If the controller is rotated 30 degrees left, the character turns 30 degrees to the left.

The problem is that the input "jitters". If I try to hold the controller perfectly still, the character's heading moves erratically within a very small cone (maybe 1 degree).

This is probably due to my shaky hands, as the controller's data is seemingly exact.

I tried filtering input by averaging data from the last X frames, but this makes input seem buttery.

My question is: How can I filter the rotational data to remove jitter without loosing precision?

Answer 1

This lag-vs-responsiveness issue is the situation with virtually all motion controllers, whether something like the Hydra, the Wii Remote, the Kinect, or the PlayStation Move.

The problem is this:

When an input stream is coming in, you're making a decision on a frame-by-frame basis about whether or not to trust the input data; whether the trends you're seeing now will have continued in the data you receive a dozen milliseconds from now. For example, if there's a sudden shift to the right this frame, you don't know whether it's a real bit of input data (and so you should act on it) or whether it's merely jitter (and so you should ignore it). Whichever you pick, if you later discover that you were wrong, you've either allowed the input jitter to make it into your game (in the first case), or introduced lag into your game (in the latter case).

There is no good solution to this. A "correct" solution to determining whether input is real or jitter requires knowing what the input stream is going to do in the future, as well as what it did in the past. We can't do that in games, for obvious reasons. So no, there is no way to filter rotational data to remove jitter without losing precision, in the context of a game which is working with input data in real-time.

I've seen a major manufacturer recommend that developers deal with this problem by having players hold down a button while rotating the control, so that the game can turn off its anti-jitter code at that point, so it's non-laggy. (I don't recommend this, but it's one approach).

I've seen a few motion input middleware libraries which deal with this problem by introducing an artificial delay in the input -- there's a quarter-second buffer that input data goes into, and your game only hears about the input a quarter second later, so that the library can smooth out the jitter for you, by knowing what happens both before and after the "present" from the game's point of view. That works great, apart from introducing a quarter second of lag to everything. But it's one way to solve the problem, and it can do an awesome job of accurately representing a motion with jitter removed, at the expense of constant lag.

But without going to that extreme, there are still some things we can do to give vastly improved behaviour, even though we know that there will always be "worst case scenarios" which behave in a non-ideal fashion.

The core insight is that we only really care about jitter when the controller is mostly stationary, and we only really care about lag when the controller is being moved. So our strategy should be to try to deal with things so that we have lag when the controller is stationary, and have jitter when the controller is in motion.

Here are two possible ways to do that:

One common approach is a "locked/unlocked" system, in which you keep track of the device's orientation, and if it doesn't change for a short while (half a second or so), you 'lock' that orientation, taking no action based upon the device's reported orientation until it differs enough to 'unlock' again. This can completely squelch orientation-based jitter, without introducing lag when the orientation is actively changing. There might be a hint of lag before the code decides it needs to switch into "unlocked" mode, but it'll be a lot better than having lag everywhere.

Another approach is to average together input data from frames. The important point here is to only average together the input data from frames where the input data was vaguely similar -- This meant that small jitters will get blurred together and softened, but larger changes don't get blurred, because their data isn't similar enough to the data from previous frames.

There are other ways to get a similar effect as well. The core insight is that you can't have both non-jitter and non-lag in your real-time game at the same time, because to do that would require knowledge of the future. So you need to pick when to bias your control's behaviour toward accepting jitter and when to bias it toward accepting lag, in order to make the overall experience as non-bad as possible.

Answer 2

It feels weird to answer my own question, but I think I've found my solution.

//Pseudo-Java

update()
{
    //deltaYaw is the change in yaw of the controller since last update
    //yawBuffer is initialized to zero, and only modified here
    //coneAngle is the stabilizing cone

    deltaYaw = getData().yaw;

    yawBuffer += deltaYaw;
    if (abs(yawBuffer) >= coneAngle)
    {
        player.yaw += (abs(yawBuffer)-coneAngle) * sign(yawBuffer);
        yawBuffer = coneAngle * sign(yawBuffer);
    }
}

Instead of modifying the player's heading directly, I simply "push" a cone of a given angle (in my case, 2.5 degrees). I made a small HTML5 demo of this technique.

Once you start pushing the cone, there is zero delay and full precision. However, if you push the cone left, and then want to aim right, you have to move across the full angle of the cone to see an effect.

So it solves the issues of temporal delay and horrible smoothing, but introduces the new problem of a movement threshold. However, if the stabilizing cone is tuned correctly, the threshold is unnoticeable.