Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

I'm trying to unproject the mouse position to get the position on the X-Z plane of a ray cast from the mouse. The camera is fully controllable by the user. Right now, the algorithm I'm using is...

    private bool getHoveredCorner(ArcBallCamera camera, System.Windows.Point m, out int tx, out int ty)
    {
        tx = -1; ty = -1;
        Ray ray = camera.unproject((float) m.X, (float) m.Y);
        Vector3 p = ray.Position + (-ray.Position.Y / ray.Direction.Y) * ray.Direction;
        float u = p.X / totalWidth;
        float v = p.Z / totalHeight;
        if(u < 0 || u > 1 || v < 0 || v > 1) return false;
        tx = (int) Math.Round(u * tileWidth);
        ty = (int) Math.Round(v * tileHeight);
        return tx >= 0 && tx <= tileWidth && ty >= 0 && ty <= tileHeight;
    }

And camera.unproject is...

    public Ray unproject(float x, float y)
    {
        Matrix m = viewProj;
        Vector3 p1 = Vector3.Unproject(new Vector3(x, y, 0), 0, 0, width, height, nearPlane, farPlane, m);
        Vector3 p2 = Vector3.Unproject(new Vector3(x, y, 1), 0, 0, width, height, nearPlane, farPlane, m);
        Vector3 dir = p2 - p1;
        dir.Normalize();
        return new Ray(p1, dir);
    }

The problem is that the projected position is "jumpy". As I make small adjustments to the mouse position, the projected point moves in strange ways. For example, if I move the mouse one pixel up, it will sometimes move the projected position down, but when I move it a second pixel, the project position will jump back to the mouse's location. The projected location is always close to where it should be, but it does not smoothly follow a moving mouse. The problem intensifies as I zoom the camera out. I'm not sure what's causing the problem, but I'm thinking it might be numerical instability?

EDIT 1: Video showing the problem

EDIT 2: A little snooping around in .NET Reflector on SlimDX.dll:

public static Vector3 Unproject(Vector3 vector, float x, float y, float width, float height, float minZ, float maxZ, Matrix worldViewProjection)
{
    Vector3 coordinate = new Vector3();
    Matrix result = new Matrix();
    Matrix.Invert(ref worldViewProjection, out result);
    coordinate.X = (float) ((((vector.X - x) / ((double) width)) * 2.0) - 1.0);
    coordinate.Y = (float) -((((vector.Y - y) / ((double) height)) * 2.0) - 1.0);
    coordinate.Z = (vector.Z - minZ) / (maxZ - minZ);
    TransformCoordinate(ref coordinate, ref result, out coordinate);
    return coordinate;
}

// ...

public static void TransformCoordinate(ref Vector3 coordinate, ref Matrix transformation, out Vector3 result)
{
    Vector3 vector;
    Vector4 vector2 = new Vector4 {
        X = (((coordinate.Y * transformation.M21) + (coordinate.X * transformation.M11)) + (coordinate.Z * transformation.M31)) + transformation.M41,
        Y = (((coordinate.Y * transformation.M22) + (coordinate.X * transformation.M12)) + (coordinate.Z * transformation.M32)) + transformation.M42,
        Z = (((coordinate.Y * transformation.M23) + (coordinate.X * transformation.M13)) + (coordinate.Z * transformation.M33)) + transformation.M43
    };
    float num = (float) (1.0 / ((((transformation.M24 * coordinate.Y) + (transformation.M14 * coordinate.X)) + (coordinate.Z * transformation.M34)) + transformation.M44));
    vector2.W = num;
    vector.X = vector2.X * num;
    vector.Y = vector2.Y * num;
    vector.Z = vector2.Z * num;
    result = vector;
}

...which seems to be a pretty standard method of unprojecting a point from a projection matrix, however this serves to introduce another point of possible instability. Still, I'd like to stick with the SlimDX Unproject routine rather than writing my own unless it's really necessary.

share|improve this question
    
Is your camera frustum plane orthogonally-aligned (that is no pitch, yaw, or roll) with respect to the tiles? Or is there arbitrary orientation / camera control in these respects? –  Nick Wiggill Dec 1 '12 at 8:36
    
@Nick Wiggill - Arbitrary orientation; the user can control/rotate the camera. –  OverMachoGrande Dec 1 '12 at 8:50
    
And to confirm, this jumping effect increases in severity from the centre of the viewport, out towards the edges? Or does it simply begin to occur with even severity at some point close to the edge? –  Nick Wiggill Dec 1 '12 at 8:55
    
@Nick Wiggill -- Sorry, the question wasn't very clear. I've updated the question. –  OverMachoGrande Dec 1 '12 at 9:47
2  
You might want to try rendering both the snapped and unsnapped positions to help you see where it's going wrong. Also what are the near and far plane values that you're using? –  Adam Sep 12 '13 at 23:14
show 4 more comments

2 Answers 2

up vote 1 down vote accepted
+300

You might want to try rendering both the snapped to grid and unsnapped positions to help you see where it's going wrong.

You should also check that the near and far plane values that you're using aren't too far apart. Most notably the bigger far/near ratio is the less accuracy you'll get both for rendering and for the calculations.

Having said that the values of 1 and 500 you said you were using seem to be well within a reasonable range to give decent accuracy to me.

When you're looking for precision issues in floating point maths, the main thing to watch out for is subtraction. If the numbers you're subtracting are close together then you'll have less precision in the result. Have a read of at least the cancellation section of http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html for more details.

share|improve this answer
add comment

Why don't you interpolate your points by "lerping"? Basically:

Vector3 Lerp(Vector3 start, Vector3 end, float percent)
{
     return (start + percent*(end - start));
}

The usage would be like Lerp(currentPosition,newPosition,time) or if you'd like it without delay Lerp(currentPosition,extraPolatedPosition,time);

share|improve this answer
1  
The question seems to be about unexpected results from unprojection, not smooth inter/extra-polation between points. No amount of tweening will help if the data is bogus. –  Lars Viklund Sep 16 '13 at 12:40
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.