I'm looking for a way to convert mouse/screen coordinates to isometric map coordinates, with the addition that the world has slopes and cliffs, and I have to be able to tell which quadrant of the tile is being pointed at by the mouse. The textures are handled by OpenGL, so I can't (easily) pick directly based on the tile sprite.

I've found several similar solutions, (e.g. Isometric Tiles Math, XNA Resources & Mouse Maps for Isometric Height Maps with the latter looking most promising) but none of them seem to quite fit my requirements.

My tiles look like this:

isometric tiles, with slopes

What algorithm or technique could I use here?

  • \$\begingroup\$ One idea is to make a second render, with a unique color for each tile, and just match color with the one the user clicked on. This is a common technique.. \$\endgroup\$ Jul 1 '15 at 18:16
  • \$\begingroup\$ How can I do that with OGL though? I don't have access to the colours? \$\endgroup\$
    – LordAro
    Jul 1 '15 at 18:21
  • 2
    \$\begingroup\$ "none of them seem to quite fit my requirements" Unless you tell us very explicitly what those requirements are, any advice we offer is likely to fall into this same situation. \$\endgroup\$
    – DMGregory
    Jul 1 '15 at 18:29
  • 2
    \$\begingroup\$ I think this will depend heavily on your current system. How do you pick flat tiles currently, and what do you do with different elevations? Your sloped tiles will have to fit into that structure somehow, so there's no concrete answer without knowing those details. \$\endgroup\$ Jul 3 '15 at 19:20
  • 1
    \$\begingroup\$ The example tile sheet looks like two copies of one tile sheet. Could the second half be safely cropped for the purposes of this question? \$\endgroup\$
    – Anko
    Jul 5 '15 at 10:08

It's not immediately obvious how you're using OpenGL. If you are just sorting your tiles on the CPU and rendering them from far to near the following method won't help. If you are using an actual isometric projection matrix and a depth buffer this may help, it's how I do picking with OpenGL and perspective projection.

Use the following code to read back the depth from the frame buffer at the given mouse coordinates.

GLfloat depth;
glReadPixels(mouse_x, mouse_y, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &depth);

Next you need to convert your mouse coords into clip space.

glm::vec4 clip;
clip.x = (mouse_x / screen_w) * 2.0f - 1.0f;
clip.y = (mouse_y / screen_h) * 2.0f - 1.0f;
clip.z = depth * 2.0f - 1.0f;
clip.w = 1.0;

You then need to multiply your clip space coords by the inverse view projection matrix.

glm::mat4 inverse_view_proj = glm::inverse(projection_matrix * view_matrix);
glm::vec4 pre_pers_div = inverse_view_proj * clip;
glm::vec3 world_position = pre_pers_div.xyz / pre_pers_div.w;

The world_position vector now contains the location of the mouse in three dimensional world coords. You can feed this into your collision detection code and see which object it is colliding with.


Summary: Make a 3d mesh exactly the size of your terrain. Then do a raytrace trough the mouse towards the terrain. This will give you the x and z coordinate of the tile. Now from the z coordinate you can calculate the y coordinate of the tile without elevation. Then you can use one of your links to calculate the tile at that position. Now some simple math will give you the quadrant.

Your terrain is probably created from a 1D or 2D array of height values. We can use this to create the 3d mesh. I am using Bullet here for some examples.

std::vector <unsigned char> heights; //This 1D vector contains the heights
width = 64;
height = 64;
//We make a shape for the terrain using the heights. We need to give bullet the width
//and height of the terrain, the heights, the scaling, the minimum height, the maximum
//height, the up axis (0:x,1:y,2:z), the data type and whether to flip quad edges.
btHeightfieldTerrainShape* terrainShape = new btHeightfieldTerrainShape(width, height, heights.data(), 1, 0, 255, 2, PHY_UCHAR, false);

//We make the Bullet rigidbody using the shape
body = new btRigidBody(0, new btDefaultMotionState(), terrainShape);

//Bullet automatically centers the body so we undo that
//Be sure to move and rotate the body in such a way that it matches your terrain
body->getWorldTransform().setOrigin(btVector3(width / 2 - 0.5f, height / 2 - 0.5f, 255/2.f));   
body->setCollisionFlags(body->getCollisionFlags() | btCollisionObject::CF_STATIC_OBJECT);

//And finally we add it to the world.

So our mesh is ready and positioned. We are ready to start raytracing. Now I am not familiar with ray tracing trough a mouse so I hope you are able to do that yourself with the help of the internet.

So once we have the coordinates we can calculate which tile is under the mouse.

Suppose the arrows in the image indicate positive directions. enter image description here

Tile column under mouse = std::floor(raycast.x);
Tile row under mouse = raycast.y - raycast.z

And that's about it I believe.

  • \$\begingroup\$ Could you give an example, or link to some sort of guide? I'm not familiar with such things \$\endgroup\$
    – LordAro
    Jul 5 '15 at 23:45
  • \$\begingroup\$ Added a little extra information. \$\endgroup\$
    – Eejin
    Jul 7 '15 at 12:33
  • \$\begingroup\$ To be honest, I'm still not sure what you mean, however, I believe I already have a method for finding the specific tile (I think it's sort of raytracing?) but I am curious what you consider to be the "simple math" to come up with the individual quadrant, as I have been unable to work it out \$\endgroup\$
    – LordAro
    Jul 11 '15 at 14:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .