How to remove jitter from motion input?

Question

I am writing a Minecraft mod that supports input from the Razer Hydra. It is a pair of motion controllers (one for each hand) that provide incredibly accurate position and rotation information.

For the purpose of this question, rotating the right controller on the Y-axis cause the player's character to look left or right (yaw), and rotating on the X-axis makes the player look up and down (pitch).

The input from the controller is mapped directly to the character's heading. If the controller is rotated 30 degrees left, the character turns 30 degrees to the left.

The problem is that the input "jitters". If I try to hold the controller perfectly still, the character's heading moves erratically within a very small cone (maybe 1 degree).

This is probably due to my shaky hands, as the controller's data is seemingly exact.

I tried filtering input by averaging data from the last X frames, but this makes input seem buttery.

My question is: How can I filter the rotational data to remove jitter without loosing precision?

Answer 1

This lag-vs-responsiveness issue is the situation with virtually all motion controllers, whether something like the Hydra, the Wii Remote, the Kinect, or the PlayStation Move.

The problem is this:

When an input stream is coming in, you're making a decision on a frame-by-frame basis about whether or not to trust the input data; whether the trends you're seeing now will have continued in the data you receive a dozen milliseconds from now. For example, if there's a sudden shift to the right this frame, you don't know whether it's a real bit of input data (and so you should act on it) or whether it's merely jitter (and so you should ignore it). Whichever you pick, if you later discover that you were wrong, you've either allowed the input jitter to make it into your game (in the first case), or introduced lag into your game (in the latter case).

There is no good solution to this. A "correct" solution to determining whether input is real or jitter requires knowing what the input stream is going to do in the future, as well as what it did in the past. We can't do that in games, for obvious reasons. So no, there is no way to filter rotational data to remove jitter without losing precision, in the context of a game which is working with input data in real-time.

I've seen a major manufacturer recommend that developers deal with this problem by having players hold down a button while rotating the control, so that the game can turn off its anti-jitter code at that point, so it's non-laggy. (I don't recommend this, but it's one approach).

I've seen a few motion input middleware libraries which deal with this problem by introducing an artificial delay in the input -- there's a quarter-second buffer that input data goes into, and your game only hears about the input a quarter second later, so that the library can smooth out the jitter for you, by knowing what happens both before and after the "present" from the game's point of view. That works great, apart from introducing a quarter second of lag to everything. But it's one way to solve the problem, and it can do an awesome job of accurately representing a motion with jitter removed, at the expense of constant lag.

But without going to that extreme, there are still some things we can do to give vastly improved behaviour, even though we know that there will always be "worst case scenarios" which behave in a non-ideal fashion.

The core insight is that we only really care about jitter when the controller is mostly stationary, and we only really care about lag when the controller is being moved. So our strategy should be to try to deal with things so that we have lag when the controller is stationary, and have jitter when the controller is in motion.

Here are two possible ways to do that:

One common approach is a "locked/unlocked" system, in which you keep track of the device's orientation, and if it doesn't change for a short while (half a second or so), you 'lock' that orientation, taking no action based upon the device's reported orientation until it differs enough to 'unlock' again. This can completely squelch orientation-based jitter, without introducing lag when the orientation is actively changing. There might be a hint of lag before the code decides it needs to switch into "unlocked" mode, but it'll be a lot better than having lag everywhere.

Another approach is to average together input data from frames. The important point here is to only average together the input data from frames where the input data was vaguely similar -- This meant that small jitters will get blurred together and softened, but larger changes don't get blurred, because their data isn't similar enough to the data from previous frames.

There are other ways to get a similar effect as well. The core insight is that you can't have both non-jitter and non-lag in your real-time game at the same time, because to do that would require knowledge of the future. So you need to pick when to bias your control's behaviour toward accepting jitter and when to bias it toward accepting lag, in order to make the overall experience as non-bad as possible.

Answer 2

I develop software that converts motion input to responsive and precise mouse input, as well as maintain a website that tries to help developers implement equally good solutions themselves. I generally advise against movement thresholds, although it depends how much responsiveness and precision players want, I'm glad that's working for you in your situation. But here I'll offer a different solution:

I use something called Soft Tiered Smoothing. The idea is that we divert input through different smoothing algorithms depending on the current magnitude of the gyro velocity (in practice, one of those smoothing algorithms is just "no smoothing"). That's the "tiered" part. The "soft" part is that we can actually smoothly split the input between different smoothing algorithms depending on how it compares to 2 thresholds.

It preserves displacement correctly and adds no lag whatsoever to quick movements.

In practice, you have two thresholds. When the magnitude of the input velocity is less than the lower threshold, we're using a simple smoothing algorithm (average across multiple frames). When it's greater than the other threshold, we use no smoothing algorithm at all. But in this case, we're still passing zeros to the lower-threshold smoothing algorithm.

When the input velocity is between the two thresholds, we split the input between the two algorithms accordingly.

Here's a snippet from the article above:

GetSoftTieredSmoothedInput(Vec2 input, float threshold1, float threshold2) {
    // this will be length(input) for vectors
    float inputMagnitude = Abs(input);

    float directWeight = (inputMagnitude - threshold1) / (threshold2 - threshold1);
    directWeight = clamp(directWeight, 0, 1);

    return GetDirectInput(input * directWeight) +
        GetSmoothedInput(input * (1.0 - directWeight));
}

GetDirectInput just returns what's given to it, but it's to show that another smoothing algorithm could be used here. GetSmoothedInput takes a velocity and returns a smoothed velocity.

With Soft Tiered Smoothing there is no smoothing applied to obviously intentional movements (above the greater threshold), there is smoothing applied to cover up small amounts of jitter, which will also affect very small movements, but when you get your thresholds right it's not very noticeable. And there's a very smooth transition between the two (without which, jitter can actually be amplified).

While the other answers are right to say that it's difficult to recognise jitter the instant an input is received, it's also true that jitter is almost always very low velocity, and the input lag that comes with smoothing is far less noticeable for low-velocity inputs.

As the article mentions, this is used in a couple of places in my open source tool JoyShockMapper, an input mapper that turns gyro input into mouse input. Even for people using other remapping tools like Steam or reWASD, some use JoyShockMapper at the same time just for its gyro controls.

This answer assumes the input is given in angular velocity (which is common with controllers that have motion controls), not absolute orientation (which it sounds like the Razer Hydra is giving you). With absolute orientation, my hope is you can use the difference between the current orientation and the previously reported orientation to get a velocity, but I don't know for sure if it'll work as well as with controllers that self-report angular velocity.

A common smoothing solution when you're dealing with an absolute position/orientation rather than velocities is to interpolate towards your goal orientation over time -- this is described in very helpful detail in this Gamasutra article. This can work with Soft Tiered Smoothing, too. You'll calculate the velocity magnitude using the difference between this input and the previous reported input. You'll apply the difference in orientation between this frame and last frame multiplied by your "directWeight" value as calculated in the snippet above. The last step is adding the smoothed input, but because of the way the interpolated orientation smoothing works, you just apply the interpolated orientation change as per normal -- it doesn't need to consider "directWeight" at all. Just set your target orientation (this is what you're interpolating towards with the smoothing described in that Gamasutra article) to whatever orientation you're getting from the device, and interpolate your orientation towards it as described in that article.

Answer 3

It feels weird to answer my own question, but I think I've found my solution.

//Pseudo-Java

update()
{
    //deltaYaw is the change in yaw of the controller since last update
    //yawBuffer is initialized to zero, and only modified here
    //coneAngle is the stabilizing cone

    deltaYaw = getData().yaw;

    yawBuffer += deltaYaw;
    if (abs(yawBuffer) >= coneAngle)
    {
        player.yaw += (abs(yawBuffer)-coneAngle) * sign(yawBuffer);
        yawBuffer = coneAngle * sign(yawBuffer);
    }
}

Instead of modifying the player's heading directly, I simply "push" a cone of a given angle (in my case, 2.5 degrees). I made a small HTML5 demo of this technique.

Once you start pushing the cone, there is zero delay and full precision. However, if you push the cone left, and then want to aim right, you have to move across the full angle of the cone to see an effect.

So it solves the issues of temporal delay and horrible smoothing, but introduces the new problem of a movement threshold. However, if the stabilizing cone is tuned correctly, the threshold is unnoticeable.