Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

im trying to find help with an issue im having moving from the XNA 3.1 framework to XNA 4.0. The shaders I wrote for my 2D game work fine in XNA 3.1, but wont even compile in 4.0.

At first I got insanely huge instruction count messages, but I switched it to compile them optimized which reduced that, but still over the limit. FXC.exe compiles them fine, but XNA 4.0 wont. I am not well versed in HLSL and could use some tips on what to do without having to redesign everything drastically.

I have tried some different approaches, [loop] attribute, etc., with mixed results. First off here is one of them that has the issue:

float4 GetLightColor(float2 position)
{
    float4 color;
    position += lightOffset; // Offset position

    // Figure out which texture this position is in, then return
    // The color in that texture
    if(position.x <= (LightSize1.x + LightSize1.z))
    {
        // Top left or bottom left
        if(position.y <= (LightSize1.y + LightSize1.w))
        {
            // Top left
            float2 offsetPosition = float2((position.x - LightSize1.x) / LightSize1.z, 
                                           (position.y - LightSize1.y) / LightSize1.w);

            color = tex2D(LightSamp1, offsetPosition);
        }
        else
        {
            // bottom left
            float2 offsetPosition = float2((position.x - LightSize3.x) / LightSize3.z, 
                                           (position.y - LightSize3.y) / LightSize3.w);

            color = tex2D(LightSamp3, offsetPosition);
        }
    }
    else
    {
        // Top right or bottom right
        if(position.y <= (LightSize2.y + LightSize2.w))
        {
            // Top right
            float2 offsetPosition = float2((position.x - LightSize2.x) / LightSize2.z, 
                                           (position.y - LightSize2.y) / LightSize2.w);

            color = tex2D(LightSamp2, offsetPosition);
        }
        else
        {
            // bottom right
            float2 offsetPosition = float2((position.x - LightSize4.x) / LightSize4.z, 
                                           (position.y - LightSize4.y) / LightSize4.w);

            color = tex2D(LightSamp4, offsetPosition);
        }
    }

    return color;
}

float4 PSBaseDoorColor(PS_INPUT input) : COLOR
{
    // Get light color from one of the 4 light textures, based on world position!
    float4 lightBase = GetLightColor(input.worldPosition);

    // I dont know why getting the texture information from the
    // regular sampler through XNA doesnt work correctly??
    float4 objectBase = tex2D(BaseSampler, input.texCoord);

    float4 ret;
    bool found = false;
    bool convertColor = false;
    int testx, testy, testz;
    float3 hsv;

    testx = round(objectBase.r * 255);
    testy = round(objectBase.g * 255);
    testz = round(objectBase.b * 255);

    for(int i = 0; i < ignoreColorsTrickNum; i++)
    {
        if(ignoreColors[i].x == testx && ignoreColors[i].y == testy && ignoreColors[i].z == testz)
        {
            found = true;
            break;
        }
    }

    if(!found)
    {
        ret = float4(2*objectBase.rgb*lightBase.rgb, objectBase.a);
    }
    else
    {
        // Do we need to convert color?
        for(int i = 0; i < 16; i++)
        {
            if(baseDoorColorChange[i].x == testx && baseDoorColorChange[i].y == testy && baseDoorColorChange[i].z == testz)
            {
                convertColor = true;
                break;
            }
        }

        if(convertColor)
        {
            hsv = rgb_to_hsv(objectBase.rgb);
            hsv.x += input.color.a;
            if ( hsv.x > 1.0 ) { hsv.x -= 1.0; }

            ret = float4(hsv_to_rgb(hsv)*pulseEffect*2, objectBase.a);
        }
        else
        {
            ret = objectBase;
        }
    }

    return ret;
}

The first issue is the GetLightColor() function which I use in several other shaders as the way lighting is designed is there are predefined lights, so its all precomputed into textures, but due to texture size limits I broke down these textures into like 2048x2048 blocks. So a player character could be standing on the four way intersection of these, necessitating 4 possible texture lookups.

XNA compiler gives warnings: Warning X4121: gradient-based operations must be moved out of flow control to prevent divergence. Performance may improve by using a non-gradient operation

While only warning, it also claims the shader is using 648 instructions, which is over the limit of 512 for ps_3_0 ! There is more to the sahder, mainly the rgb_to_hsv and back conversion functions, but those seem fine.

The second issue is I have two hardcoded int3 arrays one has like 101 elements, the other 16, of color values that I compare in the shader to the current pixel to determine if lighting should be done or not:

    for(int i = 0; i < ignoreColorsTrickNum; i++)
    {
        if(ignoreColors[i].x == testx && ignoreColors[i].y == testy && ignoreColors[i].z == testz)
        {
            found = true;
            break;
        }
    }

At first the variable was a constant, and it seemed like the compiler was unrolling the loop, so i tried switching it to a shader variable that I would just set from C# before running, which seemed to work, but then it was giving me a new compiler error:

(257,28): error X4505: maximum temp register index exceeded

Times about 60.

I also tried using float3's instead, and doing a threshold check but same result.

Perhaps a bad way to go about all this, but I dont understand why XNA 4 is having such issues with this. Even a shader that was written ages ago for ps 2.0 it reported it way over the instruction count limit.

Any help is appreciated, thanks.

share|improve this question
add comment

2 Answers

up vote 2 down vote accepted

float3 and float4 will in general use the same number of registers. Registers on GPUs are inherently all float4, with smaller vectors using up one of those. The compiler will try to be smart about it and combine smaller variables into a single register where it makes sense, but that's an optimization privilege the compiler has, not a guaranteed behavior that you can rely on as a shader author.

If-statements and loops are best avoided as much as you possibly can in shaders; when using them, try to ensure they're uniform (don't depend on per-pixel values, only uniforms). GPUs are inherently bad at branching, which is one of the costs of being able to shove 1,000+ shader cores into a single chip. Your shader code is not at all friendly to how GPUs work and the newer compiler is trying to point that out to you in its awkward way.

The gradient texture lookups are warnings because of how the hardware and math works. The way that gradients are calculated is dependent on uniform-flow. They become erroneous in some cases when in non-uniform flow control. It depends on how the texture is filtered (set outside of the shader) which is why this is a warning and not an error, but in general do all your texture accesses ahead-of-time outside of any non-uniform code-paths.

The older compilers sometimes were more permissive. There's various reasons why, but the end result is that code which gets warnings on newer versions probably wasn't working efficiently before. Don't get irritated at the newer compiler, it's just being more informative and helpful than the old one was.

For register and instruction counts, remember that D3D guarantees a minimum not a maximum. D3D might state that some limit must be a minimum of 512 but the hardware might support going up to 768 or 2048 or whatever else. The minimum is there to let you write an app that you know for sure will work on any hardware supporting that version of D3D, not to stop you from using more if you know your target hardware is newer and will support it.

share|improve this answer
    
Thanks for the info. I understand it isnt efficient at all. The one im not sure how to refactor is the for loop. Basically I have 100-116 pre-defined colors that I want the basic lighting shader to ignore. I could put those in a separate texture I suppose, but I would still need to loop through each one to compare against the current pixel, and im not sure if that is better at all, but maybe that would at least get it compiled again. Have any ideas there? –  Joshjje Jun 16 '13 at 17:16
    
A lookup texture/buffer/array. For each color, do a single query (no loop) into this can get back a value. For example, you could have a single one-component texture. Sample that texture at a coordinate corresponding to your color in some way. Store 1.0 in places you want to keep the color, 0.0 in places you don't. Multiply the return with the color. Use wrap modes, borders, and some sensible restrictions on colors to make that texture reasonably small; you may not be able to be entirely generic here. –  Sean Middleditch Jun 16 '13 at 18:36
    
Thanks, I think a Texture3D lookup will do just what I need. –  Joshjje Jun 16 '13 at 19:58
add comment

Here's some ideas to optimize that light function, off the top of my head:

  • If you just want to eliminate those compiler warnings use tex2Dlod() instead of tex2D(). I'm guessing you don't care about mip mapping since it's 2D.

  • Set up your textures to use border address mode and make the border colour black. That way you can simply read from each texture and sum the results without any if tests.

  • That leaves the problem of texture coordinates. Firstly those can be computed in the vertex shader which should be much quicker because there's usually less vertices than pixels, especially in 2D. The only limit is the number of available interpolators.

  • The maths for the texture coordinates should also be possible to simplify significantly. Calculate the position within the top left texture, ignoring the fact that it might be outside it, because the border address mode will handle it. For the top right one simply subtract 1 from the u coordinate. The bottom ones need a similar offset in the v coordinate.

share|improve this answer
    
Ya, I dont care about mipmap. OK so I could move some of that position calculation to the vertex shader, and still have to branch off on which sampler to sample from, or just sample them all and branch the result, which I think the compiler ends up doing anyways? –  Joshjje Jun 16 '13 at 17:18
    
The main problem I think is the other one though, which im not sure what to do about. Basically I have 100-116 pre-defined colors that I want the basic lighting shader to ignore. I could put those in a separate texture I suppose (right now its a giant int3 array in a Constant.fx shader include), but I would still need to loop through each one to compare against the current pixel, and im not sure if that is better at all, but maybe that would at least get it compiled again. Have any ideas there? –  Joshjje Jun 16 '13 at 17:19
    
To fix the second problem I'd store some extra data in the texture where those colours come from. The obvious location is the alpha channel if it's not used already. You can then use destination alpha, along with the appropriate blend modes to prevent lighting on areas based on the alpha channel in the render target. –  Adam Jun 16 '13 at 22:33
    
No loops. Look up the color with a texture access to find out if its a special color. You'll have to be a bit clever to avoid needing a gigantic texture; eg clustering all your special colors limiting them to a small range of values. You may to rethink what you're doing; some algorithms and effects just aren't compatible with GPUs. –  Sean Middleditch Jun 16 '13 at 22:34
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.