I'm trying to cast a ray so I can implement mouse picking. My game uses 2:1 isometric with a 3D world sim that uses AABBs. Positive X points to the bottom right, positive Y points to the bottom left, positive Z is up.

enter image description here

I currently have a screen -> world conversion function that will tell me where a mouse click intersects with Z==0:

    // Offset the screen point to include the camera position.
    float x{screenPoint.x + camera.extent.x};
    float y{screenPoint.y + camera.extent.y};

    // Calc the scaling factor going from screen tiles to world tiles.

    // Calc the world position.
    float worldX{((2.f * y) + x) * TILE_WIDTH_SCALE};
    float worldY{((2.f * y) - x) * TILE_HEIGHT_SCALE / 2.f};

Is there maybe an alternate form of this conversion that will get me a ray? I don't have a set of matrices to unproject, I'm just using these iso conversions.

Another thought I had is: the direction vector out of the camera is constant, so I could just calculate that and use it for my ray. The issue is, I don't know my camera's position so I don't know the ray's origin. My camera is very simple, it's just centered on a world-space position using the following steps:

  • Set the world-space floor (Z==0) position that you want to center the camera on.
  • Convert the world position to a point in isometric screen space.
  • Render anything within (point.x - screenWidth/2, point.y - screenHeight/2, screenWidth, screenHeight).

Edit: As requested, here's my world -> screen conversion:

    // Calc the scaling factor going from world tiles to screen tiles.

    // Convert cartesian world point to isometric screen point.
    float screenX{(position.x - position.y) * (TILE_WIDTH_SCALE / 2.f)};
    float screenY{(position.x + position.y) * (TILE_HEIGHT_SCALE / 2.f)};

    // The Z coordinate contribution is independent of X/Y and only affects the
    // screen's Y axis. Scale and apply it.
    screenY -= (position.z * Z_SCREEN_SCALE);
  • \$\begingroup\$ There's a degree of freedom in how much the vertical (z) axis is foreshortened in your projection. Can you show us your world-to-screen conversion function so we can find the appropriate foreshortening of the z axis (and correspondingly, the z component of the ray direction)? \$\endgroup\$
    – DMGregory
    Commented Jun 15, 2023 at 20:16
  • \$\begingroup\$ Sure thing, I just added it. My Z contribution is multiplied by a configurable scale, but I can set it to something specific if that's preferable. \$\endgroup\$
    – Archduke
    Commented Jun 15, 2023 at 20:49

1 Answer 1


Practically, the best way to achieve this is likely to just switch to matrices. This will minimize the amount of repetitive and error-prone code you have. You would

  1. Create a matrix that contains the same offsets and scale factors as your world-to-screen coordinate converter does.
  2. Compute the inverse of that matrix for screen-to-world.
  3. Replace both conversion functions with matrix multiplication.
  4. Produce a ray by executing screen-to-world with two different Z coordinates.

However, it is possible to work out the same ray manually (and, technically, slightly more efficient because you don't need two matrix·vector operations).

A ray can be seen as an origin and direction, or as a pair of points on a line. Your screen-to-world function already knows how to find one of those two points — the one that has z = 0. Your world-to-screen function says:

    screenY -= (position.z * Z_SCREEN_SCALE);

So, in order to find a second point that does not have z = 0 but still lands on the same screen point, we can imagine this process as:

  • Change the z to be different.
  • Adjust the other coordinates so the screen position does not change.

Step by step:

  1. Let's say that we add 1 to position.z.

  2. This will cause screenY to differ by -Z_SCREEN_SCALE.

  3. Therefore, we must change position.x and position.y so that screenY is moved by +Z_SCREEN_SCALE.

  4. We don't want to change screenX, so position.x - position.y should stay constant. The difference must stay the same, so we can conclude we should add the same quantity to both x and y.

  5. If we add the same amount to x and y then position.x + position.y changes by twice that, so if we find a quantity for position.x + position.y to change by, we should halve that.

  6. screenY = (position.x + position.y) * (TILE_HEIGHT_SCALE / 2.f), so if we invert that we get

    (position.x + position.y) = screenY / (TILE_HEIGHT_SCALE / 2.f)

  7. This is a linear equation with no constant term, so we can just substitute in the deltas:

    Δ(position.x + position.y) = ΔscreenY / (TILE_HEIGHT_SCALE / 2.f)

  8. Now let's plug in the amount we want to change:

    Δ(position.x + position.y) = Z_SCREEN_SCALE / (TILE_HEIGHT_SCALE / 2.f)

  9. From steps 4-5 we know we want equal change in x and y:

    Δx * 2 = ∆y * 2 = Z_SCREEN_SCALE / (TILE_HEIGHT_SCALE / 2.f)

  10. Now let's apply basic algebraic simplification to that formula:

    Δx * 2 = ∆y * 2 = Z_SCREEN_SCALE * 2.f / TILE_HEIGHT_SCALE


Now putting it all together, we've got the offset for our second point from the first point, which is also the direction-vector of the ray.

(Δx, Δy, Δz) = (

Since the length of a direction vector doesn't matter (except when you want a normalized one), you can multiply the whole thing by TILE_HEIGHT_SCALE and get a different equivalent one:

(Δx, Δy, Δz) = (

Either way, since I chose to start with increasing Z, this is a direction vector that points upward, out of the screen. You can just negate it to point inward.

Also, I may have made some math error — getting signs backwards is a classic. But I have tried to lay out the logic by which I got there, so you can check it or apply it to a different starting point yourself.

And if you use matrices instead, there's a lot fewer chances for error.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .