0
\$\begingroup\$

in my desperation while trying to achieve this (I am a HLSL noob) i try to ask my question here. I am modifying an old water shader for the game "MTA San Andreas" (the multiplayer modification). MTA uses HLSL and comes with some builtin predefined variables and functions. The water shader itself is already modified by me, but now i want to add shore fading to it. Objects slightly below water surface should be visible, but objects deeper in the water should be completely invisible and that effect should be configurable. I found a water shader that has shore fading included, but it was covered within very complex other things in a very large file. I managed to extract most of the necessary information (i think) but i get an error message in the game: "Invalid PS 3_0 input semantic "POSITION" in water.fx 139,23" I know that this line 138 has no errors and the only reason why the game throws this error is actually line 268... Debugging HLSL is a pain, horrible language. And there may be much more wrong stuff. This is the water shader:

https://pastebin.com/VDFxS7RX

I started my modifications in line 268, where objdepth seems to be "wrong" and causes the compile error:

float objdepth = input.position.z/input.position.w;
float nonlinearobjdepth = objdepth;
objdepth = 1.0/max(1.0-objdepth, 0.000000001);
float planardepth = tex2D(SamplerDepth, input.textureCoords.xy).r;
float depth = tex2D(SamplerDepth, input.textureCoords.xy).r;
if (nonlinearobjdepth>depth)
{
    depth = planardepth;
}

float scenedepth = depth;
      planardepth = 1.0/max(1.0-planardepth,0.000000001);
      depth = 1.0/max(1.0-depth,0.000000001);
float depthfact = (depth-objdepth)*WaterParameters1.z;
      depthfact = depthfact*WaterParameters1.x;
      depthfact = depthfact/(depthfact*0.50+1.0);
if (scenedepth>0.99999) depthfact = 1.0;
float backside = saturate(WaterParameters1.w*10.0);
      depthfact*= 1.0-backside;
float shorefade = (planardepth-objdepth);
      shorefade = shorefade*WaterParameters1.x;
      shorefade = saturate(13.40*shorefade*1000.0 - 0.05);
      depthfact*= shorefade;
finalColor.a *= shorefade;

return finalColor;
\$\endgroup\$
0
\$\begingroup\$

The best would be working with linear depths for this effect. You will need two depths.

  1. Depth of water mesh you are rendering. input.position.w should be the linear depth value of the water mesh.
  2. Depth of the underlying geometry that was rendered beforehand. You will read it as a texture from screen coordinates. If you have a native depth buffer available for that render pass, you will need to convert that to linear depth. The reason is that you will want the "water depth fog" to look like linearly increasing.

This is a function to convert native depth buffer value to linear depth:

inline float getLinearDepth(in float z, in float near, in float far)
{
    float z_n = 2.0 * z - 1.0;
    float lin = 2.0 * far * near / (near + far - z_n * (near - far));
    return lin;
}

You will want to calculate the difference of these two depths and make the water more transparent when the difference is small. For example fading out a blue water color could be this:

float water_depth = input.position.w; // linear depth spanning from near plane to far plane
float scene_depth = tex2D(SamplerDepth, input.textureCoords.xy).r;
scene_depth = getLinearDepth(scene_depth, my_nearplane, my_farplane); // if your texture contains native depth buffer, use this to convert to linear
float fade_strength= 0.1; // how strong the fade effect is
float depth_difference = saturate(abs(water_depth - scene_depth) * fade_strength);
float4 watercolor = float4(0,0,1,depth_difference); // transparency fade

I suggest to simplify your code, and output different pieces to debug, like output linear depths, depth difference and see how they look like on screen if something doesn't look right from the start. Good luck!

\$\endgroup\$
  • \$\begingroup\$ Thanks for your reply! What i have seen now is that input.position.z and w cant be used for some magic reason. What i have to use is the vposition. There were some more issues in the code and i found a solution to actually make it work tonight, i will insert my solution as answer and try your solution now. My solution has some issues that seem to need fixing. \$\endgroup\$ – Einheit-101 Sep 16 at 10:58
  • \$\begingroup\$ Oh, i forgot: Unfortunately there is no way in MTA to debug shaders, they only throw an error in debug console when they fail to compile due to a critical error. This makes the job not easier. \$\endgroup\$ – Einheit-101 Sep 16 at 11:16
  • \$\begingroup\$ Oh, right system semantics like POSITION might not be readable with shader model 3.0, I have switched to sm 5.0 a long time ago.. In that case you can output the position a second time from the vertex shader to a custom TEXCOORD semantic and read from that, which is a bit wasteful. And yes, debugging shaders on PC is a pain, there is no proper debug support that works everywhere. :( \$\endgroup\$ – János Turánszki Sep 16 at 21:00
  • \$\begingroup\$ Yes, i hope MTA team will start introducing SM 5.0 with DX11, which should be done next year. Your approach had some issues, for example the water becomes fully transparent when the camera tilted towards it, the solution that i found afterwards looks better, i simply copied a piece of code from an ENBseries mod. It has rendering issues when using Jetpack and aircraft, but i think that is a GTA engine bug and cannot be fixed. I opened a bug report. youtube.com/watch?v=GfH0AfNn6BA \$\endgroup\$ – Einheit-101 Sep 16 at 22:22
0
\$\begingroup\$

I have found a possible solution, but its not "final" and i will try János solution next. The problem is that my solution produces strange artifacts that may be related to GTA San Andreas rendering problems. This is the full shader as it is "working" now with its issues:

//-- Include some common stuff
#include "mta-helper.fx"

float Time : Time;
texture skyBoxTexture1;
texture skyBoxTexture2;
float3 skyRotate = float3(0, 0, 0);
float fadeValue = 0;
float4 deepness = 0.5; //this value must be adjusted depending on flight height / wearing jetpack (??)
float4 ScreenSize = float4(0,0,0,0);
texture causticTexture;
texture reflectionTexture;
texture refractionTexture;
texture normalTexture;
texture gDepthBuffer : DEPTHBUFFER;

float flowSpeed = 0.0;
float reflectScale = 0.0;
float refractScale = 0.0;
float reflectionStrength = 0.0;
float refractionStrength = 0.0;
float causticStrength = 0.0;
float4 waterColor = float4(0.0, 0.0, 0.0, 0.0);
float brightness = 1.0;
float dayTime = 1.0;
float3 sunPos = float3(0, 0, 0);
float4 sunColor = float4(0.0, 0.0, 0.0, 0.0);
float specularSize = 4;
float waterShiningPower = 1;
float fogStart = 50;
float fogEnd = 550;

///////////////////
// SAMPLE STATES //
///////////////////

sampler CausticSampler = sampler_state
{
    Texture = <causticTexture>;
    MinFilter = Linear;
    MagFilter = Linear;
    MipFilter = Linear;
    AddressU = Wrap;
    AddressV = Wrap;
};

sampler ReflectionSampler = sampler_state
{
    Texture = <reflectionTexture>;
    MinFilter = Linear;
    MagFilter = Linear;
    MipFilter = Linear;
    AddressU = Mirror;
    AddressV = Mirror;
};

sampler RefractionSampler = sampler_state
{
    Texture = <refractionTexture>;
    MinFilter = Linear;
    MagFilter = Linear;
    MipFilter = Linear;
    AddressU = Mirror;
    AddressV = Mirror;
};

sampler NormalSampler = sampler_state
{
    Texture = <normalTexture>;
    MinFilter = Linear;
    MagFilter = Linear;
    MipFilter = Linear;
    AddressU = Wrap;
    AddressV = Wrap;
};

sampler SamplerDepth = sampler_state
{
    Texture = (gDepthBuffer);
    MinFilter = Point;
    MagFilter = Point;
    MipFilter = None;
    AddressU = Clamp;
    AddressV = Clamp;
};

samplerCUBE SkyCubeSampler1 = sampler_state
{
    Texture = <skyBoxTexture1>;
    MinFilter = Linear;
    MagFilter = Linear;
    MipFilter = Linear;
    AddressU = Clamp;
    AddressV = Clamp;
};

samplerCUBE SkyCubeSampler2 = sampler_state
{
    Texture = <skyBoxTexture2>;
    MinFilter = Linear;
    MagFilter = Linear;
    MipFilter = Linear;
    AddressU = Clamp;
    AddressV = Clamp;
};

float3x3 eulRotate(float3 Rotate)
{
    float cosX,sinX;
    float cosY,sinY;
    float cosZ,sinZ;

    sincos(Rotate.x,sinX,cosX);
    sincos(-Rotate.y,sinY,cosY);
    sincos(Rotate.z,sinZ,cosZ);

//Euler extrinsic rotations 
//http://www.vectoralgebra.info/eulermatrix.html


        float3x3 rot = float3x3(
        cosY * cosZ + sinX * sinY * sinZ, -cosX * sinZ,  sinX * cosY * sinZ - sinY * cosZ,
        cosY * sinZ - sinX * sinY * cosZ,  cosX * cosZ, -sinY * sinZ - sinX * cosY * cosZ,
        cosX * sinY,                       sinX,         cosX * cosY
    );

return rot; 
}

struct VertexInputType
{
    float3 position : POSITION;
    float3 normal : NORMAL0;
    float2 textureCoords : TEXCOORD0;
};

struct PixelInputType
{
    float4 position : POSITION;
    float2 textureCoords : TEXCOORD0;
    float4 reflectionPosition : TEXCOORD1;
    float4 refractionPosition : TEXCOORD2;
    float3 skyTextureCoordinate : TEXCOORD3;
    float Depth : TEXCOORD4;
    float3 worldPosition : TEXCOORD5;
    float3 lightDirection : TEXCOORD6;
    float3 worldNormal : TEXCOORD7;
    float4 vposition : TEXCOORD8;

};

////////////////////////////////////////////////////////////////////////////////
// Vertex Shader
////////////////////////////////////////////////////////////////////////////////
PixelInputType WaterVertexShader(VertexInputType input)
{
    PixelInputType output;
    matrix reflectProjection;
    matrix refractProjection;

    // Calculate the position of the vertex against the world, view, and projection matrices.
    output.position = MTACalcScreenPosition(input.position);
    output.worldPosition = MTACalcWorldPosition(input.position);
    output.lightDirection = normalize(gCameraPosition - sunPos);
    output.worldNormal = MTACalcWorldNormal(input.normal);

    // Store the texture coordinates for the pixel shader.
    output.textureCoords = input.textureCoords;

    // Create the view projection world matrix for reflection.
    reflectProjection = mul(gWorldViewProjection, gWorld);
    reflectProjection = mul(gWorld, reflectProjection);

    // Calculate the input position against the refractProjection matrix.
    output.reflectionPosition = mul(float4(input.position, 1), reflectProjection);

    // Create the view projection world matrix for refraction.
    refractProjection = mul(gWorldViewProjection, gWorld);
    refractProjection = mul(gWorld, refractProjection);

    // Calculate the input position against the refractProjection matrix.
    output.refractionPosition = mul(float4(input.position, 1), refractProjection);

    float4 vertexPosition = mul(float4(input.position, 1), gWorld);

    // compute the eye vector 
    float4 eyeVector = vertexPosition - gViewInverse[3];            
    output.skyTextureCoordinate = mul(eulRotate(skyRotate), eyeVector.xyz);

    output.Depth = output.position.z;
    float4 pos = float4(input.position.x,input.position.y,input.position.z,1.0);
    float4 tpos = mul(pos, gWorldViewProjection);
    output.vposition = tpos;
    return output;
}



////////////////////////////////////////////////////////////////////////////////
// Pixel Shader
////////////////////////////////////////////////////////////////////////////////
float4 WaterPixelShader(PixelInputType input) : COLOR0
{
    float2 refractTexCoord;
    float2 reflectTexCoord;
    float3 skyBoxReflectTexCoord;
    float3 skyBoxTexCoord;
    float4 normalMap;
    float3 normal;
    float4 reflectionColor;
    float4 refractionColor;
    float4 causticColor;
    float4 skyColor;
    float4 color;
    float timer = (Time/12) * flowSpeed;

    // Move the position the water normal is sampled from to simulate moving water. 

    // Calculate the projected refraction texture coordinates.
    reflectTexCoord.x = input.reflectionPosition.x / input.reflectionPosition.w / 2.0f + 0.5f;
    reflectTexCoord.y = -input.reflectionPosition.y / input.reflectionPosition.w / 2.0f + 0.5f;
    refractTexCoord.x = input.refractionPosition.x / input.refractionPosition.w / 2.0f + 0.5f;
    refractTexCoord.y = -input.refractionPosition.y / input.refractionPosition.w / 2.0f + 0.5f;
    skyBoxReflectTexCoord.x = -input.skyTextureCoordinate.x / 1 / 2.0f + 0.5f;
    skyBoxReflectTexCoord.y = -input.skyTextureCoordinate.y / 1 / 2.0f + 0.5f;
    skyBoxReflectTexCoord.z = input.skyTextureCoordinate.z / 1 / 2.0f + 0.5f;
    // Sample the normal from the normal map texture.
    float2 NormalTex = input.textureCoords;
    NormalTex.y = NormalTex.y + timer;
    normalMap = tex2D(NormalSampler, NormalTex);

    // Expand the range of the normal from (0,1) to (-1,+1).
    normal = (normalMap.xyz * 2.0f) - 1.0f;

    // Re-position the texture coordinate sampling position by the normal map value to simulate the rippling wave effect.
    reflectTexCoord = reflectTexCoord + (normal.xy * reflectScale);
    refractTexCoord = refractTexCoord + (normal.xy * refractScale);
    skyBoxReflectTexCoord = skyBoxReflectTexCoord + (normal * reflectScale);
    // Sample the texture pixels from the textures using the updated texture coordinates.
    float2 CausticTex = input.textureCoords;
    CausticTex.y = CausticTex.y  + (sin(CausticTex.y * 10) * 0.01) + timer;
    causticColor = tex2D(CausticSampler, CausticTex);

    float4 skyColor1 = texCUBE(SkyCubeSampler1, 1 - skyBoxReflectTexCoord.yzx);
    float4 skyColor2 = texCUBE(SkyCubeSampler2, 1 - skyBoxReflectTexCoord.yzx); 
    float4 finalSkyColor = (skyColor2 * fadeValue) + (skyColor1 * (1 - fadeValue));

    reflectionColor = tex2D(ReflectionSampler, reflectTexCoord) * reflectionStrength;
    refractionColor = tex2D(RefractionSampler, refractTexCoord) * refractionStrength;

    // Using Blinn half angle modification for performance over correctness
    float3 lightRange = normalize(normalize(gCameraPosition - input.worldPosition) - input.lightDirection);
    float specularLight = pow(saturate(dot(lightRange, normal)), specularSize * 2);
    float4 specularColor = float4(sunColor.rgb * specularLight, 1);
    specularColor += pow(saturate(dot(lightRange, input.worldNormal)), specularSize / 2) / 2;
    specularColor *= normalMap.g * normalMap.g;

    // Combine the reflection and refraction results for the final color.
    color = lerp(reflectionColor * reflectionStrength, refractionColor * refractionStrength, 2.0f);
    color *= causticColor * causticStrength;
    color += finalSkyColor * waterColor * reflectionStrength;

    float distanceFog = saturate((input.Depth - fogStart)/(fogEnd - fogStart));
    float4 finalColor = lerp(float4(color.rgb, 1), float4(reflectionColor.rgb/2, 1), distanceFog);
    finalColor.rgb += specularColor.rgb * normalMap * waterShiningPower * 0.7;
    finalColor.rgb *= brightness;
    finalColor.a *= waterColor.a;
    finalColor = float4(MTAApplyFog(finalColor.rgb, input.worldPosition), finalColor.a);
    float diffGray = saturate( 0.1 + (( finalColor.r +finalColor.g+finalColor.b )/3 ) * dayTime);
    finalColor = lerp( float4( diffGray, diffGray, diffGray, finalColor.a ), finalColor, saturate( dayTime ));

    float2 txcoord;
    txcoord.xy = (input.vposition.xy / input.vposition.w) * float2(0.5, -0.5) + 0.5;
    txcoord.xy += 0.5 * float2(ScreenSize.y, ScreenSize.y*ScreenSize.z);

    float objdepth = input.vposition.z / input.vposition.w;
    objdepth = 1.0/max(1.0-objdepth, 0.000000001);
    float planardepth = tex2D(SamplerDepth, txcoord.xy).r;
    planardepth = 1.0/max(1.0-planardepth,0.000000001);

    float shorefade = planardepth-objdepth;
    shorefade = saturate(shorefade*deepness);
    finalColor.a *= shorefade;

    return finalColor;
}

////////////////////////////////////////////////////////////////////////////////
// Technique
////////////////////////////////////////////////////////////////////////////////
technique WaterTechnique
{
    pass pass0
    {
        ZEnable = true;
        ZWriteEnable = true;
        ZFunc = 2;
        DepthBias = 0.000001;
        SlopeScaleDepthBias = 2;
        VertexShader = compile vs_3_0 WaterVertexShader();
        PixelShader = compile ps_3_0 WaterPixelShader();
    }
}

// Fallback
technique fallback
{
    pass P0
    {
        // Just draw normally
    }
}
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.