How do I convert a 2D movement (from mouse) to a displacement along a 3d axis?

Question

I'm making a small 3D scene editor (that only allow to modify objects positions).

Each object is draw with a 3d X/Y/Z axis :

How it works : user choose an axis (X, Y or Z) by left clicking on it, then, while holding left click, he move the mouse (to move object in the axis direction) . picking is already ok, but i'd like to know how much the object should be translated for the selected axis, depending 2D mouse movement.

For the example, on this picture, selecting X axis and moving mouse left-right should give a bigger displacement than moving it up-down (almost zero for this one)

It must be really simple but i cannot figure it out.

Input should be : mouse displacement since first left click(2D vector), selected axis vector (3d vector), modelview / projection matrix.

Output : amount (float or double)

Answer 1

How do you want this to behave? There are different ways to do this.

A simple option is to just move the object by some fixed number of world space units for each screen space unit (pixel, say) that the mouse moves.

Another option is to take the vector of mouse movement and project it onto the axis of movement through the normal projection/camera matrices. Project the mouse movement vector via the inverse viewport, projection, and camera matrices (give it an initial Z of the near view plane distance, you should end up with an offset on the near plane). This gives you the mouse movement vector in world space. Project that vector onto the desired axis. This gives you the movement in world space. The object will move to where the user pointed, independent of what orientation the object is at. If the axis is mostly vertical, it takes vertical movement to move it.

You can also just take the start and end mouse point in screen space, project both, and from that calculate the world space movement vector. I don't, but you mighty find that easier.

Another option is to take a power of the mouse movement, so faster movements cause larger offsets.

Using a variant of the first option, you can also grab the pointer so that the user can continuously move his mouse (picking it up and back down). That's especially popular for rotation and scale controls.

Answer 2

I would probably project the selected axis endpoints into 2d space, subtract them to produce a 2d vector. Then take the dot product of the normalized vector of 2d mouse motion with the normalized first vector to determine the portion of movement that should be applied.

So basically, compute 2 vectors:

Va = normalized difference of the projected axis endpoints

Vb = normalized vector created from the mouse movement delta x and y

Va.Vb = |Va||Vb|cos@

Va.Vb / (|Va|Vb|) = cos@ -- which is the component of mouse movement that should be applied along the axis.

Just another option.