Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

I am setting an HLSL effect variable in the following way in a number of places.

extern ID3D10EffectVectorVariable* pColour;

pColour = pEffect->GetVariableByName("Colour")->AsVector();

pColour->SetFloatVector(temporaryLines[i].colour);

In one of the places it is set in a loop, each line in the vector temporaryLines has a D3DXCOLOR variable associated with it. The most annoying thing about this problem is that it actually works on rare occasions, but most of the time it doesn't. Are there any known issues with this kind of code?

Here it works:

void GameObject::Draw(D3DMATRIX matView, D3DMATRIX matProjection)
{
device->IASetInputLayout(pVertexLayout);

mesh.SetTopology();//TODO should not be done multiple times

// select which vertex buffer and index buffer to display
UINT stride = sizeof(VERTEX);
UINT offset = 0;
device->IASetVertexBuffers(0, 1, mesh.PBuffer(), &stride, &offset);
device->IASetIndexBuffer(mesh.IBuffer(), DXGI_FORMAT_R32_UINT, 0);

pColour->SetFloatVector(colour);

// create a scale matrix
D3DXMatrixScaling(&matScale, scale.x, scale.y, scale.z);

// create a rotation matrix
D3DXMatrixRotationYawPitchRoll(&matRotate, rotation.y, rotation.x, rotation.z);

// create a position matrix
D3DXMatrixTranslation(&matTranslation, position.x, position.y, position.z);

// combine the matrices and render
matFinal = 
    matScale        * 
    matRotate       * 
    matTranslation  * 
    matView * matProjection;
pTransform->SetMatrix(&matFinal._11); 
pRotation->SetMatrix(&matRotate._11);    // set the rotation matrix in the effect
pPass->Apply(0);
device->DrawIndexed(mesh.Indices(), 0, 0);   //input specific
}

Here is occasionally works:

void BatchLineRenderer::RenderLines(D3DXMATRIX matView, D3DXMATRIX matProjection)
{
device->IASetInputLayout(pVertexLayout);

device->IASetPrimitiveTopology(D3D10_PRIMITIVE_TOPOLOGY_LINESTRIP);

// select which vertex buffer and index buffer to display
UINT stride = sizeof(LINE);
UINT offset = 0;
device->IASetVertexBuffers(0, 1, &pBuffer, &stride, &offset);
device->IASetIndexBuffer(iBuffer, DXGI_FORMAT_R32_UINT, 0);

allLines = temporaryLines.size();

for(int i = 0; i < allLines; i++)
{
    pColour->SetFloatVector(temporaryLines[i].colour); // in the line loop too?

    // combine the matrices and render
    D3DXMATRIX matFinal = 
        temporaryLines[i].scale * 
        temporaryLines[i].rotation * 
        temporaryLines[i].position * 
        matView * matProjection;

    pTransform->SetMatrix(&matFinal._11); 
    pRotation->SetMatrix(&temporaryLines[i].rotation._11);    // set the rotation matrix in the effect

    pPass->Apply(0);

    device->DrawIndexed(2, 0, 0);
}

temporaryLines.clear();
}

the effect file:

float4x4 Transform;    // a matrix to store the transform
float4x4 Rotation;     // a matrix to store the rotation transform
float4   LightVec   = {0.612f, 0.3535f, 0.612f, 0.0f};    // the light's vector
float4   LightCol   = {1.0f,   1.0f,    1.0f,   1.0f};    // the light's color
float4   AmbientCol = {0.3f,   0.3f,    0.3f,   1.0f};    // the ambient light's color
float4   Colour;

// a struct for the vertex shader return value
struct VSOut
{
    float4 Col : COLOR;    // vertex normal
    float4 Pos : SV_POSITION;    // vertex screen coordinates
};

// the vertex shader
VSOut VS(float4 Norm : NORMAL, float4 Pos : POSITION)
{
    VSOut Output;

    Output.Pos = mul(Pos, Transform);    // transform the vertex from 3D to 2D

    Output.Col = AmbientCol;    // set the vertex color to the input's color

    float4 Normal = mul(Norm, Rotation);

    Output.Col += saturate(dot(Normal, LightVec)) * LightCol * Colour; // add the diffuse and passed in light

    return Output;    // send the modified vertex data to the Rasterizer Stage
}

// the pixel shader
float4 PS(float4 Col : COLOR) : SV_TARGET
{
    return Col;    // set the pixel color to the color passed in by the Rasterizer Stage
}

// the primary technique
technique10 Technique_0
{
    // the primary pass
    pass Pass_0
    {
        SetVertexShader(CompileShader(vs_4_0, VS()));
        SetGeometryShader(NULL);
        SetPixelShader(CompileShader(ps_4_0, PS()));
    }
}
share|improve this question
    
This questions seems like it might receive better answers on stack overflow instead of game development. –  Jcubed Jan 3 '12 at 22:37
    
I believe we've established D3D is passing the value through correctly and that it's most likely the value of the normals on the vertices at either end of each line that are being used to light the vertex. Lighting a line (which doesn't really have a single normal direction that makes sense) in the same manner as a triangle is resulting in the unwanted behaviour. –  Roger Perkins Jan 5 '12 at 7:15
add comment

2 Answers 2

up vote 1 down vote accepted
+150

Without being able to see it running (or the rest of the code) it's hard to say, but are you sure it's actually what you think failing here? That value is only used in the lighting equation where it's contribution is based on the angle between the normal of the vertex and the lighting vector. It could be the normal value on the line vertex is such that it breaks. Have you tried just setting the output colour of the VS (Output.Col) to just be equal to "Colour" to rule out any other possibilities? That way the value will just be set directly and will validate it's this that's broken. With that set, also try setting a specific value in the code rather than pull the value out of the line structure to ensure that's valid.

Other than that, there's any number of reasons it could be broken, e.g. the vertex layout might not exactly match the actual structure correctly, etc.

I'm not sure what the state of PC shader debugging is like as I haven't had much call to use them (e.g. PIX, NVidia or ATI tools), but they might also be able to let you see better what's going on.

share|improve this answer
    
setting the Output.Col to the colour does work. So why exactly could this be breaking? –  Asher Einhorn Jan 3 '12 at 10:01
    
You're multiplying "Colour" by LightCol (which is a constant so we can most likely rule that out) and the dot product of the Normal with LightVec. LightVec is also a constant, so most likely either Norm (the input normal from the vertex structure) is wrong/invalid, or Rotation, the matrix used to transform Norm into Normal is broken in some way. I'd hazard a guess as this is a line you're rendering that the vertex normal isn't something good (like 0,0,0) maybe, the concept of a normal on a vertex at the end of a line is a weird concept anyway. Does the line need/want shading like this? –  Roger Perkins Jan 3 '12 at 23:26
    
not necessarily, but I don't like shying away from things that seem to be working incorrectly, even if I'm not going to use them like that in the end. Also, on the occasion that the lines do draw in colour they have quite a nice effect to them, so I might end up keeping them like that. –  Asher Einhorn Jan 4 '12 at 9:57
    
Well we've proven that the SetFloatVector value is getting through correct (by directly setting just that value as the output colour), so DX is working just fine here, it's the lighting shader logic. I'm not sure what value for a normal makes sense to set on a vertex that's on either end of a line, but it's going to be the value of the normal you've set that's causing the behaviour. –  Roger Perkins Jan 5 '12 at 7:11
    
you're absolutely right by the way. I wrote a solid shader for the lines and they're working now. It was a multiplication by 0 problem in the shader. Also, just as a side note, this was the last little glitch in a demo I was making. Now to apply for my first games industry job! exciting! –  Asher Einhorn Jan 5 '12 at 16:44
show 1 more comment

Set your FloatVector before you set the Input Layout.

I had the same problem once:

http://stackoverflow.com/questions/2936472/setmatrix-does-not-copy-all-values-to-hlsl

share|improve this answer
    
useful stuff, thank you! –  Asher Einhorn Jan 5 '12 at 16:45
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.