HLSL texture not reading from register S1

Question

I made a simple post processing shader, that draws scanlines. This all works perfectly. I wanted to make it a bit more interesting by applying a shadowmask instead so I wanted to pass a texture to the shader. From my understanding the Spritebatch sets Texture Register(S0), so I set my shadowmask texture to Texture Register(s1).

When applying the shader is seems that both S0 and S1 contain the texture I plan to draw. I have tried to set parameters in different ways- without succes. How can I pass the Texture?

My HLSL code:

sampler ScreenTextureSampler: register (s0); // This is the texture that SpriteBatch will try to set before drawing
sampler ShadowSampler: register (s1); // This is set in GraphicsDevice.Textures[1];

int colorlines = 2;
int shadowlines = 1;


//------------------------ PIXEL SHADER ----------------------------------------

float4 PixelShaderFunction(float4 pos : SV_POSITION, float4 color1 : COLOR0, float2 texCoord : TEXCOORD0) : SV_TARGET0
{
    float4 color = tex2D(ScreenTextureSampler, texCoord.xy);
    float4 maskcolor = tex2D(ShadowSampler, texCoord.xy);

    float ypos = pos.y;
    float scanline = (ypos) % (colorlines + shadowlines);
    float intensity = scanline < colorlines ? 1 : 0.20f;

    //comment out one of these two return statements...
    return maskcolor*intensity; //note this return has the exact same result
    return color*intensity;     //as this one! even though the samplers point at different registers.

}

technique Postprocessing
{
    pass Pass1
    {
        PixelShader = compile ps_4_0  PixelShaderFunction();
    }
}

This shader is loaded in an Effect called Assets.GameArt.PostProcessor. I have a shadowmask Texture2D in Assets.GameArt.shadowTexture.

My C#, Monogame code looks like this:

        gd.SetRenderTarget(null);

        sb.Begin(SpriteSortMode.BackToFront, BlendState.AlphaBlend, SamplerState.PointClamp, DepthStencilState.None, RasterizerState.CullNone, Assets.GameArt.PostProcessor, Resolution.getTransformationMatrix());

        gd.Textures[1] = Assets.GameArt.shadowTexture;


        sb.Draw(_gameRenderTarget, Vector2.Zero, _camera.Viewport, Color.White);

        sb.End();

gd points to the GraphicsDevice and sbis the spriteBatch. I first render my scene to a _gameRenderTarget.

Why isn't the Texture passed via the S1 register (or: why do the texture samplers both return the same result?)

Answer 1

It turns out that this is due to optimisation of the .fx compiler. When only one sampler is actually used (regardless of the number of samplers being declared) the effect is compiled to be optimized to only read from the first register.

So: With this declaration:

sampler ScreenTextureSampler: register (s0) // This is the texture that SpriteBatch will try to set before drawing
{
    Texture = (Diffuse);
    magfilter = LINEAR;
    minfilter = LINEAR;
    mipfilter = LINEAR;
    AddressU = wrap;
    AddressV = wrap;
};

Texture2D Shadow;
sampler ShadowSampler: register (s1)
{
    Texture = (Shadow);
    magfilter = LINEAR;
    minfilter = LINEAR;
    mipfilter = LINEAR;
    AddressU = wrap;
    AddressV = wrap;
};

The results depend on what is done with the values. So this code:

float4 PixelShaderFunction(float4 pos : SV_POSITION, float4 color1 : COLOR0, float2 texCoord : TEXCOORD0) : SV_TARGET0
{
    float4 color = tex2D(ScreenTextureSampler, texCoord.xy);
    float4 maskcolor = Shadow.Sample(ShadowSampler, texCoord.xy);

    return maskcolor;
}

Doesn't use the color variable and therefore also not two samplers. This causes the maskcolor to get optimized and read from the first register (s0) instead.

float4 PixelShaderFunction(float4 pos : SV_POSITION, float4 color1 : COLOR0, float2 texCoord : TEXCOORD0) : SV_TARGET0
{
    float4 color = tex2D(ScreenTextureSampler, texCoord.xy);
    float4 maskcolor = Shadow.Sample(ShadowSampler, texCoord.xy);

    return color * maskcolor;
}

Uses both of the samplers and yields the correct result.

Conclusion: The effect compiler optimizes and discards any values that do not contribute to the return value. The samplers do not read from fixed registers, but are used in the order they are used in the pixel shader (so the first call to a sampler reads from the s0, the second call reads from s1 and so on).

Answer 2

You were adding Assets.GameArt.shadowTexture as Texture2D to your graphicdevice but there are no any texture associated in your HLSL file. You need add Texture2D to your pixel shader file and use your sampler to sample your texture2D. I suggest to render it without spritebatch and go one level lower and create your own quad with vertices, then render it with a pixel shader like as following:

Texture2D Texture : register(t0);
SamplerState Sampler : register(s0);

float4 main(PS_In IN) : SV_TARGET
{
    return Texture.Sample(Sampler, IN.uv);
}