# Rendering differences between ps_1_3 and ps_2_a

I am trying to update a shader from ps_1_3 inline assembly to ps_2_a HLSL. However the HLSL shader causes things to render differently when using ps_2_a. When using my HLSL conversion with ps_1_3 it works same way as with the inline assembly.

Original assembly

Sampler[0] = (DiffuseMapSampler);
{
ps_1_3
tex t0
#ifdef _POINTLIGHT_
+mov r0.a, v0.a
mul_x4 r0, t0, r0
#else
mul_x4 r0, t0, v0
#endif
mul_x2 r0.a, r0.a, t0.a
#endif
};


Sampler and VS_OUTPUT

struct VS_OUTPUT
{
float4 Pos  : POSITION0;
float2 Tex0 : TEXCOORD0;
#endif
float4 Color  : COLOR0;
float Fog   : FOG;
};

texture DiffuseMap;
sampler DiffuseMapSampler = sampler_state
{
Texture = (DiffuseMap);
MipFilter = LINEAR;
MinFilter = LINEAR;
MagFilter = LINEAR;
MipMapLodBias = 0;
};


My HLSL translation

float4 basicPixelShaderNoShadow(VS_OUTPUT VsOut) : COLOR
{
float4 diffuseMap = tex2D(DiffuseMapSampler, VsOut.Tex0);
float4 outCol;
#ifdef _POINTLIGHT_
outCol.rgb = VsOut.Color.rgb + VsOut.Color.rgb;
outCol.a = VsOut.Color.a;
outCol = (diffuseMap * outCol) * 4;
#else
outCol = (diffuseMap * VsOut.Color) * 4;
#endif

outCol.a = (outCol.a * diffuseMap.a) * 2;
#endif
return outCol;
}


And this is the result in game For some reason when using ps_2_a the colours are oversaturated.

Edit: HLSL that fixed the issue:

float4 vertexColor = float4(VsOut.Color.rgb, VsOut.Color.a*2);
float4 diffuseMap = tex2D(DiffuseMapSampler, VsOut.Tex0);

float4 outCol = diffuseMap * vertexColor * 2;

outCol.a *= 2 * diffuseMap.a;
#endif

return outCol;


dont quite understand why, but it works :)

I haven't (yet) found definitive confirmation of this (and it's likely lost in the mists of time, in ancient documentation that no longer exists), but SM1.3 is clearly clamping intermediates to 1.0 whereas SM2 is not.

So, for example, where you have:

outCol.rgb = VsOut.Color.rgb + VsOut.Color.rgb;


And:

outCol = (diffuseMap * outCol) * 4;


If the result of any of these exceeds 1.0, SM1.3 is clearly clamping it. To replicate the SM1.3 behaviour in SM2 you could use e.g. the clamp or max intrinsics.

• I tried using min, clamp and saturate to clamp it to a maximum of 1, but it always looked the same. The only thing I could find about clamping on the r registers is the range table on the bottom of this article. There it states those have minimum range of -1/+1 in 1.3 and -8/+8 in 1.4. I tested the HLSL in 1.4 and it works there, so dont think it is clamping due to range here Jan 23 at 22:35
• @Mate391 - I suggest checking the generated assembly for the SM2 version. You're definitely getting an intermediate clamped, I'm 100% positive on that, but the trick is to find which intermediate. It's not always obvious from the HLSL. I'd pay particular attention to the HLSL that should generate those mul_x4 lines because it's a high chance it's in there. Jan 24 at 0:58
• I played around with it some more and finally got a solution. I added that solution to the question. Still dont really understand why, but it works :D Jan 24 at 11:53
• @Mats391 Solutuons should go in Answers, not in Questions. Jan 24 at 13:02