# Why does my triangle ray intersection test always return -1.0f?

I'm trying to get the mouse click position on a terrain geometry, but multiple times in each frame; doing it on the CPU was way too slow, so I'm trying to do it with a compute shader. I can't seem to get a valid result back. I guess it's somehow related to my raydirection/rayorigin vectors, because I tried 4 different written DirectX HLSL solutions, but none of them works for my case; though, I did have to modify them. Anyway, this is my code to generate the ray vectors:

XMFLOAT3 RayOriginFloat = m_Camera->GetPosition(); //The eye position of the camera. I use this vector in the compute shader.

XMVECTOR worldPos;
DirectX::BoundingFrustum frustum;

// I use the DirectXMath XMVector3Unproject method to project the mouse coord to worldspace.
ScreenToWorld(m_mouseX, m_mouseY, &worldPos);

XMVECTOR RayDir = worldPos - RayOrigin;
RayDir = XMVector3Normalize(RayDir); //And finally the normalized direction vector


And this is the HLSL code:

cbuffer RayBuffer
{
float3 RayOrigin;
float3 RayDir;
matrix view;
};

struct ComputeInputType
{
float3 position : POSITION;
float2 tex : TEXCOORD;
float3 normal : NORMAL;
};

struct ComputeOutputType
{
float distance;
float3 position;
};

static const uint ThreadSize = 32;

StructuredBuffer<ComputeInputType> BufferIn : register(t0);
RWStructuredBuffer<ComputeOutputType> BufferOut : register(u0);

{
float result;
float3 vertices[3];
float3 edge[2];

{

edge[0] = vertices[1].xyz - vertices[0].xyz;
edge[1] = vertices[2].xyz - vertices[0].xyz;

float3 normal = normalize(cross(edge[0], edge[1]));
float b = dot(normal, RayDir);

float3 w0 = RayOrigin - vertices[0].xyz;
float a = -dot(normal, w0);
float t = a / b;

result = t;

float3 p = RayOrigin + t * RayDir;
float uu, uv, vv, wu, wv, inverseD;
uu = dot(edge[0], edge[0]);
uv = dot(edge[0], edge[1]);
vv = dot(edge[1], edge[1]);

float3 w = p - vertices[0].xyz;
wu = dot(w, edge[0]);
wv = dot(w, edge[1]);
inverseD = uv * uv - uu * vv;
inverseD = 1.0f / inverseD;

float u = (uv * wv - vv * wu) * inverseD;
if (u < 0.0f || u > 1.0f)
{
result = -1.0f;
}

float v = (uv * wu - uu * wv) * inverseD;
if (v < 0.0f || (u + v) > 1.0f)
{
result = -1.0f;
}

}
}


The result is always -1.0f. I don't get it; I might have made a big mistake, but I can't find where.

Why does my triangle ray intersection test always return -1.0f?

• What algorithm do you use? Möller-Trumbore? Jun 23 '17 at 16:07
• Yes, Möller-Trumbore. Jun 23 '17 at 16:15