I am currently looking through the Introduction to 3D Game Programming with Direct X 11 by Frank Luna and I'm trying to implement the ray-triangle intersections. I already have a ray firing when the mouse is clicked and from what I can tell it is the correct direction as I have made the camera move to the direction I click. The book uses XNA math which from what I can tell got replaced by DirectXMath? The equivalent functions in the book are TriangleTests::Intersects found here https://msdn.microsoft.com/en-us/library/windows/desktop/hh855922(v=vs.85).aspx

But it doesn't seem to work in my application and it doesn't find any intersection.

Does anyone know why? Here is my attempt

for (int i = 0; i < terrain->terrainIndices.size() / 3; i++) {
    int i0 = terrain->terrainIndices[i * 3 + 0];
    int i1 = terrain->terrainIndices[i * 3 + 1];
    int i2 = terrain->terrainIndices[i * 3 + 2];

    //vertices for single tri
    XMVECTOR v0 = XMVectorSet(terrain->terrainVertices[i0].Pos.x, terrain->terrainVertices[i0].Pos.y, terrain->terrainVertices[i0].Pos.z, 0.0f);
    XMVECTOR v1 = XMVectorSet(terrain->terrainVertices[i1].Pos.x, terrain->terrainVertices[i1].Pos.y, terrain->terrainVertices[i1].Pos.z, 0.0f);
    XMVECTOR v2 = XMVectorSet(terrain->terrainVertices[i2].Pos.x, terrain->terrainVertices[i2].Pos.y, terrain->terrainVertices[i2].Pos.z, 0.0f);

    //if ray intersects this triangle
    if (TriangleTests::Intersects(rayOrigin, rayDirection, v0, v1, v2, dist) || TriangleTests::Intersects(rayOrigin, rayDirection,
        v0, v2, v1, dist)) {
        stringstream ss;
        ss << "INTERSECTION FOUND" << endl;
        return true;
    else {
        stringstream ss;
        ss << "NO INTERSECTION" << endl;

Edit: I have also tried this. The loop is the same when I loop over my terrain triangles for calculating normals.

for (int i = 0; i < 512; i++) {

        for (int j = 0; j < 512; j++) {

            int index1 = (j * 512) + i;
            int index2 = (j * 512) + (i + 1);
            int index3 = ((j + 1) * 512) + i;

            int index4 = ((j + 1) * 512 + i);
            int index5 = ((j * 512 + (i + 1)));
            int index6 = ((j + 1) * 512 + (i + 1));

            //vertices of triangle
            Vertex vertex1 = terrain->terrainVertices.at(index1);
            Vertex vertex2 = terrain->terrainVertices.at(index2);
            Vertex vertex3 = terrain->terrainVertices.at(index3);

            Vertex vertex4 = terrain->terrainVertices.at(index4);
            Vertex vertex5 = terrain->terrainVertices.at(index5);
            Vertex vertex6 = terrain->terrainVertices.at(index6);

            //each vertex as XMVECTOR
            XMVECTOR point1 = XMVectorSet(vertex1.Pos.x, vertex1.Pos.y, vertex1.Pos.z, 0.0f);
            XMVECTOR point2 = XMVectorSet(vertex2.Pos.x, vertex2.Pos.y, vertex2.Pos.z, 0.0f);
            XMVECTOR point3 = XMVectorSet(vertex3.Pos.x, vertex3.Pos.y, vertex3.Pos.z, 0.0f);

            XMVECTOR point4 = XMVectorSet(vertex4.Pos.x, vertex4.Pos.y, vertex4.Pos.z, 0.0f);
            XMVECTOR point5 = XMVectorSet(vertex5.Pos.x, vertex5.Pos.y, vertex5.Pos.z, 0.0f);
            XMVECTOR point6 = XMVectorSet(vertex6.Pos.x, vertex6.Pos.y, vertex6.Pos.z, 0.0f);
            //if ray intersects this triangle
            if (TriangleTests::Intersects(rayOrigin, rayDirection, point1, point2, point3, dist) || TriangleTests::Intersects(rayOrigin, rayDirection,
                point1, point3, point2, dist)
                || TriangleTests::Intersects(rayOrigin, rayDirection, point4, point5, point6, dist)
                || TriangleTests::Intersects(rayOrigin, rayDirection, point4, point6, point5, dist)) {
                stringstream ss;
                ss << "INTERSECTION FOUND" << endl;
                return true;

Edit: The ray I'm firing is normalized, is this a problem for this function? Am I even using it correctly?

Edit: Nvm the function only works when the vector is normalized. Still can't seem to get any intersection. The rayOrigin and rayDirection seem to be fine. The origin is the cameras position when I click and the direction is a normalized vector of the direction i click in.

Edit: TriangleTests::Intersects is new for DirectXMath. This functionality is not available in XNAMath 2.x. Similar functionality for XNAMath can be found in the DirectX SDK Collision sample.

Does this have anything to do with the fact I'm not translating the object or rays into a different space? I.e. World space/object space

Edit: This is how I am getting calculating the ray direction

float pointX, pointY;
    XMMATRIX projectionMatrix, viewMatrix, inverseViewMatrix, worldMatrix, translateMatrix, inverseWorldMatrix;
    XMVECTOR direction, origin, rayOrigin, rayDirection;
    bool intersect, result;

    projectionMatrix = XMLoadFloat4x4(this->projectionMatrix);
    viewMatrix = XMLoadFloat4x4(this->viewMatrix);

    // Move the mouse cursor coordinates into the -1 to +1 range.
    pointX = ((2.0f * (float)mouseX) / (float)screenWidth) - 1.0f;
    pointY = (((2.0f * (float)mouseY) / (float)screenHeight) - 1.0f) * -1.0f;

    pointX = pointX / XMVectorGetX(projectionMatrix.r[0]);
    pointY = pointY / XMVectorGetY(projectionMatrix.r[1]);

    // Get the inverse of the view matrix.
    inverseViewMatrix = XMMatrixInverse(nullptr, viewMatrix);

    //click on screen and camera will rotate on the world Y axis to the clicked point.
    direction = 
        XMVectorSet((pointX * XMVectorGetX(inverseViewMatrix.r[0])) + (pointY * XMVectorGetX(inverseViewMatrix.r[1])) + XMVectorGetX(inverseViewMatrix.r[2]),
                    (pointX * XMVectorGetY(inverseViewMatrix.r[0])) + (pointY * XMVectorGetY(inverseViewMatrix.r[1])) + XMVectorGetY(inverseViewMatrix.r[2]),
                    (pointX * XMVectorGetZ(inverseViewMatrix.r[0])) + (pointY * XMVectorGetZ(inverseViewMatrix.r[1])) + XMVectorGetZ(inverseViewMatrix.r[2]), 0.0f);

    camera->turnCameraToPoint(XMFLOAT3(camera->getPosition().x + XMVectorGetX(direction),
        camera->getPosition().y + XMVectorGetY(direction),
        camera->getPosition().z + XMVectorGetZ(direction)));

    std::stringstream ss;

    // Get the origin of the picking ray which is the position of the camera.
    origin = XMVectorSet(camera->getPosition().x, camera->getPosition().y, camera->getPosition().z, 0.0f);

    // Get the world matrix and translate to the location of the sphere.
    worldMatrix = XMLoadFloat4x4(this->worldMatrix);
    //translateMatrix = terrain->getObjectWorld();
    //worldMatrix = XMMatrixMultiply(worldMatrix, translateMatrix);

    // Now get the inverse of the translated world matrix.
    //inverseWorldMatrix = XMMatrixInverse(nullptr, worldMatrix);

    // Now transform the ray origin and the ray direction from view space to world space.
    rayOrigin = XMVectorSet(camera->getPosition().x, camera->getPosition().y, camera->getPosition().z, 0.0f);
    //rayOrigin = XMVector3TransformCoord(origin, worldMatrix);
    //rayDirection = XMVector3TransformCoord(direction, worldMatrix);
    rayDirection = direction;

    // Normalize the ray direction.
    rayDirection = XMVector3Normalize(rayDirection);

    ss << " clicked direction: " << XMVectorGetX(rayDirection) << " y: " << XMVectorGetY(rayDirection) << " z: " << XMVectorGetZ(rayDirection) << endl;


    // Now perform the ray-triangle intersection test.
    float distance = 0.0f;
    intersect = terrainIntersection(rayOrigin, rayDirection, distance);

    if (intersect == true)
        // If it does intersect then set the intersection to "yes" in the text string that is displayed to the screen.
        std::stringstream ss5;

        ss5 << "INTERSECTION!! - " << "dist: " << distance << endl;
        // If not then set the intersection to "No".
        std::stringstream ss5;
        ss5 << "NO INTERSECTION..." << endl;


Edit: I have managed to get a valid intersection from the method, but there is something incorrect as it is now returning a valid intersection for anywhere I click on the screen, and the direction of the ray is producing incorrect results. I have tried to make sure that the ray, ray origin and the vertices of the triangle are all in the same space by doing

XMMATRIX translationMatrix = XMMatrixTranslation(camera->getPosition().x, camera->getPosition().y, camera->getPosition().z);
XMMATRIX finalWorld = translationMatrix * worldMatrix * viewMatrix * projectionMatrix;
rayOrigin = XMVector3TransformCoord(origin, finalWorld);

Each vertex is also in the same space

        finalWorld = XMMatrixIdentity();
        translationMatrix = XMMatrixTranslation(vertex3.Pos.x, vertex3.Pos.y, vertex3.Pos.z);
        finalWorld = translationMatrix * world * view * projection;
        point3 = XMVector3TransformCoord(point3, finalWorld);

This now produces a valid intersection

clicked direction: -0.799399 y: -0.0989474 z: -0.592597 INTERSECTION FOUND INTERSECTION!! - dist: -0

Although this allows me to click anywhere on the screen and the intersection is deemed valid, even when I am not clicking anywhere on the terrain. Does anyone have any idea why this is happening? The distance returned from the TriangleTests::Intersects method is always 0 or -0

  • \$\begingroup\$ Are your vertices, ray origin, and ray in the same space (local, world, view)? If not, you won't get hits. Typically you'd transform your ray/direction from world space back to 'local' space for the terrain, but it would also work to transform the vertices into world space. rayDirection needs to be normalized as well. Note that the latest Collision sample that uses DirectXMath is also available if you want to play with it. \$\endgroup\$ Aug 11, 2017 at 20:13
  • \$\begingroup\$ @ChuckWalbourn thanks for the reply. I believe everything is in world space yes, although I am not sure. The ray origin and ray have world space coordinates when printed which are correct and so do the vertices. When I multiply these with the world matrix using XMVector3TransformCoord, the coordinates of the ray direction are incorrect. I will have a play with the example in your comment, but do you have any other guidance about making sure the ray and the vertices are in the same space? Thanks! - Edit, I have added into the post how I am getting the fired ray. \$\endgroup\$
    – user
    Aug 13, 2017 at 19:17
  • \$\begingroup\$ I don't seem to be doing anything with the world/view/projection matrix with the ray/vertices, I assumed they were already in world space as the coordinates were correct. Do I have to do something specific here? \$\endgroup\$
    – user
    Aug 13, 2017 at 19:28
  • \$\begingroup\$ I have updated the post with my latest efforts. I have managed to get a valid intersection from the intersects method, but it is returning true no matter where I click on the screen. Any guidance? \$\endgroup\$
    – user
    Aug 13, 2017 at 21:27

1 Answer 1


I have actually fixed the issue I was having. I needed to transform the vectors into the same spaces using

XMVector3Transform(origin, ray->getObjectWorld());

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .