0
\$\begingroup\$

I have what appears to be a depth or order rendering problem which I have never seen happen before. My terrain and water are rendered with my displacement shader, and at the moment all other models are rendered with my standard shader, and I am using deferred rendering. Now, as you can see in the images, I have a sphere that is half cut off, as though d3d thinks that the water is somehow in front of the sphere, which it is not. If I move down to water level, you can see the whole sphere. If I render the same sphere with my displacement shader, there is no problem at all. If I go into my displacement shader class, and cut out the hull and domain shader stages, then the terrain and water start to do the weird z buffer issue that the sphere is having. Is there something conceptual anyone can think of without me having to post hundreds of lines of code? However, happy to post code if needed.

Thank you

enter image description here

enter image description here

UPDATE: I've narrowed it down to the domain shader - the line that physically changes the output triangles' position. When I comment out this line, it solves the problem, although then I have no tessellation. Do I need to do some kind of manual depth writing after changes the position?

Domain Shader:

cbuffer cbDomainInfo
{
    float3 g_vEyePosition;
    float _padding;
};

struct PatchTess
{
    float flEdges[ 3 ] : SV_TessFactor;
    float flInside : SV_InsideTessFactor;
};

struct DomainInput
{
    float4 vPosition    : POSITION;
    float2 vTexcoord    : TEXCOORD0;
    float3 vNormal      : NORMAL;
    float3 vViewVec     : TEXCOORD1;
    float flAlpha       : ALPHA;
};

struct DomainOutput
{
    float4 vPosition    : SV_POSITION;
    float2 vTexcoord    : TEXCOORD0;
    float3 vNormal      : NORMAL;
    float3 vViewVec     : TEXCOORD1;
    float flAlpha       : ALPHA;
};

Texture2D g_texDispMap;
SamplerState g_samSampler;

[ domain( "tri" ) ]
DomainOutput DS( PatchTess input, float3 uvwCoord : SV_DomainLocation, const OutputPatch<DomainInput, 3> patch )
{
    DomainOutput output;

    output.vPosition = uvwCoord.x * patch[ 0 ].vPosition + uvwCoord.y * patch[ 1 ].vPosition + uvwCoord.z * patch[ 2 ].vPosition;
    output.vNormal = uvwCoord.x * patch[ 0 ].vNormal + uvwCoord.y * patch[ 1 ].vNormal + uvwCoord.z * patch[ 2 ].vNormal;
    output.vTexcoord = uvwCoord.x * patch[ 0 ].vTexcoord + uvwCoord.y * patch[ 1 ].vTexcoord + uvwCoord.z * patch[ 2 ].vTexcoord;
    output.vViewVec = uvwCoord.x * patch[ 0 ].vViewVec + uvwCoord.y * patch[ 1 ].vViewVec + uvwCoord.z * patch[ 2 ].vViewVec;
    output.flAlpha = uvwCoord.x * patch[ 0 ].flAlpha + uvwCoord.y * patch[ 1 ].flAlpha + uvwCoord.z * patch[ 2 ].flAlpha;

    output.vNormal = normalize( output.vNormal );

    float flMipInterval = 20.f;
    float flMipLevel = clamp( ( distance( output.vPosition, float4( g_vEyePosition, 1.f ) - flMipInterval ) ) / flMipInterval, 0.f, 6.f );

    float flHeightMap = g_texDispMap.SampleLevel( g_samSampler, output.vTexcoord, flMipLevel ).r;

    //TODO: Pass in height scale via cbuffer
    //This next line causes the problem:
    output.vPosition += float4( ( 20.f *( flHeightMap - 1.f ) ) * output.vNormal, 1.f );

    return output;
}
\$\endgroup\$
  • \$\begingroup\$ RenderDoc is your friend. renderdoc.org \$\endgroup\$ – Syntac_ Mar 21 at 8:04
  • \$\begingroup\$ @Syntac_ thank you for the suggestion - I just tried it out, RenderDoc crashes when loading the capture. \$\endgroup\$ – T. Elliott Mar 21 at 8:57
  • \$\begingroup\$ We will indeed need to see your code to diagnose this precisely. To avoid dumping hundreds of lines to trawl though, you should instead work on first creating a Minimal, Complete, Verifiable Example, making a copy of your project and cutting out absolutely everything that isn't required to demonstrate the artifact, so we can reproduce the problem in the simplest case. This will help isolate the issue faster. \$\endgroup\$ – DMGregory Mar 21 at 11:49
  • \$\begingroup\$ From what you have explained there seems to be a fundamental difference in way your calculating your vertices in your vertex shader. Is there a difference in the matrices involved between the 2 shaders used. Or even any order differences between the 2? \$\endgroup\$ – ErnieDingo Mar 21 at 19:27
  • \$\begingroup\$ @ErnieDingo I have managed to narrow the down the problem to the domain shader - updating the output vertex position (based on heightmap) seems to be interfering with the other shaders. When commenting out the line that changes the position, everything seems to work ok. Do I need to do some kind of manual depth writing? (Please see updated post) \$\endgroup\$ – T. Elliott Mar 27 at 7:08
0
\$\begingroup\$

DMGregory was right - setting the w coordinate to 1.f was causing the problem. I was able to fix it by changing it to 0.f, or alternatively only changing output.vPosition.xyz. If anyone has an explanation as to perhaps why this is I'd be interested to hear about it.

Thanks to DMGregory and ErnieDingo for the help.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.