Mouse input accumulates different number points at different frame rates

Question

I'm working on a basic Vulkan renderer that's presently relying on Win32 raw mouse input for camera pitch and yaw. It's all very basic, but what I've noticed is that at higher frame rates, I'm accumulating fewer "points" of movement with comparable gestures of the mouse than I do at lower frame rates. This manifests as fewer degrees of rotation for the camera for a given amount of mouse input.

A WM_INPUT message will propagate the lLastX and lLastY values out via my event system untouched. The window receiving these messages is DPI aware (I've got 150% scaling on my development machine).

auto const *input = reinterpret_cast<::RAWINPUT *>(_raw_message);
switch(input->header.dwType) {
    case RIM_TYPEMOUSE: {
        auto const &mouse = input->data.mouse;
        if(mouse.lLastX != 0 || mouse.lLastY != 0) {
            EventBroker::emit<MouseMoveEvent>(
                mouse.lLastX,
                mouse.lLastY
            );
        }
    }
    default:
        break;
}

And the game code just adds those values to the camera's tracked pitch and yaw angles, accounting for frame time and a base movement speed. This camera setup is basically identical to that from the LearnOpenGL camera chapter.

void Demo::update() {
    auto const mouse_speed = _cam_data.mouse_speed * Timekeeper::frametime();
    _cam_data.pitch += _mouse.y_offset * mouse_speed;
    _cam_data.yaw   += _mouse.x_offset * mouse_speed;

    _mouse.y_offset = 0;
    _mouse.x_offset = 0;

    if(_cam_data.pitch > 89.9f)       { _cam_data.pitch = 89.9f;  }
    else if(_cam_data.pitch < -89.9f) { _cam_data.pitch = -89.9f; }

    auto const cos_yaw   = std::cosf(math::radians(_cam_data.yaw));
    auto const sin_yaw   = std::sinf(math::radians(_cam_data.yaw));
    auto const cos_pitch = std::cosf(math::radians(_cam_data.pitch));
    auto const sin_pitch = std::sinf(math::radians(_cam_data.pitch));

    _cam_data.forward.x = cos_yaw * cos_pitch;
    _cam_data.forward.y = sin_pitch;
    _cam_data.forward.z = sin_yaw * cos_pitch;

    _cam_data.forward = math::normalize(_cam_data.forward);
    _cam_data.side = math::normalize(
        math::cross(_cam_data.forward, Vec4::unit_y)
    );
    _cam_data.up = math::cross(_cam_data.side, _cam_data.forward);

    auto const kb_speed = _cam_data.kb_speed * Timekeeper::frametime();
    if(_kb.w)      { _cam_data.pos += _cam_data.forward * kb_speed; }
    else if(_kb.s) { _cam_data.pos -= _cam_data.forward * kb_speed; }
    if(_kb.a)      { _cam_data.pos -= _cam_data.side    * kb_speed; }
    else if(_kb.d) { _cam_data.pos += _cam_data.side    * kb_speed; }

    _persp_camera.orient(
        _cam_data.pos,
        _cam_data.forward,
        _cam_data.side,
        _cam_data.up
    );

    _vp_matrices.view = _persp_camera.view_matrix();
    _vp_matrices.proj = _persp_camera.proj_matrix();

    BufferTools::update_buffer(
        _vp_ubos[Swapchain::image_index()],
        &_vp_matrices
    );
}

In this case, _cam_data.mouse_speed happens to be 5.0f. Nothing too fancy. I expect to achieve a similar visual movement for a similar mouse input regardless of frame rate.

So what am I fundamentally misunderstanding here? (Arithmetic, you say? Of course.) Shouldn't sampling once per frame, then multiplying by the frame time account for the different values one gets for the increased polling rate? Or is there more OS faffery behind the scenes? I've used raw mouse input several times in past projects, but it appears I only ever tested with vsync on. Or I was doing something different, but I can't track that difference down myself.

EDIT: Just to drive the arithmetic point home, I see that if I multiply the WM_INPUT mouse positions by a fixed value (rather than the frame time) higher frame rates create faster camera movement. Surely there's a simple(r?) solution to divorce the frame rate from the rotation rate.

Answer 1

The problem is here:

auto const mouse_speed = _cam_data.mouse_speed * Timekeeper::frametime();

Multiplying by the frame duration is appropriate when you want to take a speed of movement and integrate it to get a displacement in position over that time interval.

But _mouse.x_offset and _mouse.y_offset are not speeds, they're already displacements measured over that time interval.

Assuming your hand is moving the mouse at a constant speed, over a short frame it will move a smaller distance, and over a long frame it will move a longer distance, so x_offset and y_offset are already scaled in proportion to the elapsed time, just by the nature of the way you're measuring them.

When you multiply them by delta time, you're "double dipping", so the result you get is the true speed multiplied by the square of the frame interval, yielding smaller-than-desired values on short frames and longer-than-desired values on long frames. (This is effectively the antiderivative of displacement... if that were a thing)

You'll note that while the OpenGL tutorial you linked does multiply camera speed by delta time when animating the camera on an auto-advancing orbit, they drop this when they introduce mouse control. Instead they scale the mouse input by a different variable, sensitivity, which is not scaled by delta time.

I know you said that multiplying by a constant speed without delta time seemed to have the opposite problem, but I'd recommend trying again, under a more controlled test (say, make a line of tick marks on a sheet of paper and run your mouse over them in time with a stopwatch). Subtle topics like control feel are very susceptible to the observer effect, where if we expect to see a particular outcome, we'll tend to perceive that outcome, whether or not it's verifiably there.

Here, you had a good reason to expect that not multiplying by delta time would result in exaggerated movement at high frame rates, because that's exactly what happens for most other kinds of input. So that prior belief can make evidence suggestive of faster movement jump out as confirmatory, or even lead you to subconsciously move your mouse faster to get the expected result.