0
\$\begingroup\$

I have a pixel shader that downsamples a texture from one size to another. The downsample shader uses a nested 'for' loop to traverse the uv coordinates. When the for loop executes more than four times( [unroll(8)] for example ) the graphics device is lost and all hell brakes loose...Here is shader code:

void DownsamplePass_Pixel_Shader( in float4 Position : POSITION0,
           in float4 PositionSS : VPOS, out float4 vEnergy : COLOR0 )
{
    float width = LastMipInfo.x;
    float height = LastMipInfo.y;
    float mip = LastMipInfo.z;
    float mult = LastMipInfo.w;
    float4 vPos = PositionSS;
    float4 vColor = 0;
    int ncPixels = 0;

    // depending on the size of the visibility buffer( texture to downsample )
    //      it may be necessary to sample more than a 2x2 block of pixels( if the size is greater than twice the target )
    //      this is only necessary for the first pass
    if( bFirstDownsamplePass )
    {
        [unroll(8)]
        for( int y = 0; y < mult; ++y )
        {
            [unroll(8)]
            for( int x = 0; x < mult; ++x )
            {
                float2 vCoord = float2( ( vPos.x * 2 * mult ) / width + x * ( 1 / width ), ( vPos.y * 2 * mult ) / height + y * ( 1 / height ) );
                float4 vTexel = tex2Dlod( g_SamplerMip, float4( vCoord, 0, mip ) );
                if( all( vTexel.rgb == 0 ) )
                    continue;

                vColor.rgb += vTexel.rgb * tex2D( g_SamplerMultiplier, vCoord ).rgb;                        
                ncPixels++;
            }
        }
    }
    else
    {
        float2 vCoords[ 4 ];
        vCoords[ 0 ] = float2( ( vPos.x * 2 ) / width, ( vPos.y * 2 ) / height );    
        vCoords[ 1 ] = float2( vCoords[ 0 ].x + ( 1 / width ), vCoords[ 0 ].y );
        vCoords[ 2 ] = float2( vCoords[ 0 ].x, vCoords[ 0 ].y + ( 1 / height ) );
        vCoords[ 3 ] = float2( vCoords[ 1 ].x, vCoords[ 2 ].y );

        // calc average energy
        float4 vColor = 0;
        int ncPixels = 0;
        for( int i = 0; i < 4; ++i )
        {
            float4 vTexel = tex2Dlod( g_SamplerMip, float4( vCoords[ i ], 0, mip ) );
            if( all( vTexel.rgb == 0 ) )
                continue;

            vColor.rgb += vTexel.rgb;

            ncPixels++;
        }
    }

    vEnergy = float4( vColor.rgb / max( ncPixels, 1 ), 1 );
}

I could just define each uv coordinate but I'm interested in knowing why this problem occurs!

\$\endgroup\$
2
\$\begingroup\$

I can think of two obvious possibilities:

  1. You've hit the TDR timeout because the shader is taking ages to run. You can work round this locally by adjusting the timeout in the registry. You probably shouldn't adjust that setting for other people though - the general solution is to break the draw call up into multiple smaller draw calls - resizing is usually quicker when done in multiple intermediate steps instead of all in one go anyway.

  2. You've found some sort of driver bug. If adjusting the timeout didn't help then try it on a different graphics card, or the Reference Rasterizer. It might also help if you could post a shader that can be compiled by someone else.

As a side note efficient downsampling shaders use the hardware bilinear filter to do a quarter as many texture reads as the obvious algorithm.

\$\endgroup\$
1
  • \$\begingroup\$ Thanks...I fixed the problem by removing the unroll() functions and adding conditions to the for loop: for( int y = 0; y < mult && y < 8; ++y ) ... I am however trying to use bilinear filtering as you suggested! \$\endgroup\$
    – P. Avery
    Jan 2 '14 at 23:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.