3
\$\begingroup\$

As a quick background, this is in the context of a custom game engine I'm writing in C# on top of the SharpDX framework. In the engine, when I render UI, I do so by creating a canvas object with its owner render target texture. I render to that texture, wrap this texture on the canvas quad, then render the canvas to the world. As a note, I am using forward rendering, not deferred.

But, the problem is, the canvas renders the texture, but the actual rendering of my UI test panel, to the texture, generates a transparent patch rather than the desired panel. Here's a screen-shot of what I'm describing:

enter image description here

The back buffer was cleared to solid red for now just for testing, and I've attempted to render a panel to the center of it with a test image. As you can see, the red back buffer was rendered to the canvas nicely, but the test image I rendered to the panel is rendering transparently. I use the same shaders/hlsl files to render both the canvas and the panel on the canvas, so I'm skeptical that this is the source of the problem.

Here's a code snippet, for where I render my canvas array, that might help:

//start preparing matrix buffer
viewMatrix = activeCamera.GetViewMatrix();
MatrixBuffer matrices = new MatrixBuffer();
matrices.viewMatrix = Matrix.Transpose(viewMatrix);
matrices.projMatrix = Matrix.Transpose(DXEnv.orthoProjectionMatrix);

//change context if needed, and draw using the shader
if (activeShader != ShaderID.TEXTURE_UNLIT)
{
            textureUnlitShader.SetActive(DXEnv.device, DXEnv.context);
            activeShader = ShaderID.TEXTURE_UNLIT;
}

for (int i=0 ; i< canvases.Length; i++)
{
    //clear render depth/render target & set the render-to-texture target
    DXEnv.context.ClearDepthStencilView(DXEnv.depthView, D3D11.DepthStencilClearFlags.Depth, 1, 0);               
    DXEnv.context.ClearRenderTargetView(canvases[i].renderTarget, new Color4(1.0f, 0.0f, 0.0f, 1.0f));
    DXEnv.SetNewRenderTarget(canvases[i].renderTarget);

    //recursively render to the new texture
    textureUnlitShader.SetTextureResource(DXEnv.device, DXEnv.context, canvases[i].source);
    RenderPanelRecursive(canvases[i].root, matrices);

    canvases[i].CalculateModelMatrix(transforms[i]);
    matrices.modelMatrix = Matrix.Transpose(canvases[i].modelMatrix);

    //set texture to the new resource and reset render view, then render canvas
    DXEnv.SetNewRenderTarget(DXEnv.renderView); //TODO: batchh up canvas rendering to minimize context switch
    textureUnlitShader.SetTextureResource(DXEnv.device, DXEnv.context, canvases[i].shaderTexture); //this is the line i tried commenting out
    textureUnlitShader.UpdateShader(DXEnv.device, DXEnv.context, matrices, canvases[i]);
}

The call to textureUnlitShader.UpdateShader(...) are most likely not the issue I've found. I tried commenting the line textureUnlitShader.SetTextureResource(..., canvases[i].shaderTexture) that was set in the above code, and, as expected, instead of the render-to-texture buffer being rendered to the canvas, my test image was rendered to the canvas. But interestingly enough, the hole remained. Which leads me to believe that the panel is not actually being rendered to the canvas's texture properly. Rather, it seems to be being rendered somehow separately. Which leads me to believe that the most likely culprit is the way I'm implementing the render to texture technique that is somehow off. Here's a screenshot of when that happens:

enter image description here

My call to render panel recursive looks a bit like the render call for the canvas, but it is as follows:

    private void RenderPanelRecursive(Panel p, MatrixBuffer matrices)
    {
        //render lowest level to highest level to preserve order
        if ( (p.active == true) && (p.renderable == true))
        {
            ImagePanel castP = p as ImagePanel;

            castP.CalculateModelMatrix();
            matrices.modelMatrix = Matrix.Transpose(castP.modelMatrix);
            textureUnlitShader.UpdateShader(DXEnv.device, DXEnv.context, matrices, castP); //perhaps build this into the shader instead?
        }

        //recurse down further if more children exist
        if (p.currentNumberControls != 0)
        {
            for (int i = 0; i < p.currentNumberControls; i++)
            {
                RenderPanelRecursive(p.children[i], matrices);
            }
        }

    }

I don't believe much could have gone wrong with panel itself. The matrix constant buffer seems to be being written to properly, as the panel renders in the right spot on the canvas. And the vertex-position/uv-coord constant buffer is identical to the structure of the canvas's vertex-position/uv-coord constant buffer. To make sure of this, I've read over the section a couple of times now and ran through it with some break points set to make sure that these buffers we're being written correctly. Nothing turned up out of the ordinary.

Which brings me back to my statement earlier, that there is probably something to the way in which I set up my render-to-texture that is going wrong.

Thanks in advance for any help you can provide.

\$\endgroup\$
  • 1
    \$\begingroup\$ I had problems understanding your problem. I think you one time labeled the transparent thing "quad" and later the bigger rectangle "quad". I suggest you check the terminology when referring to "quad". Sounds like you have some state set when it should not be set like Ernie suggested with the blend states or z-buffer. \$\endgroup\$ – Benedikt S. Vogler May 29 '17 at 21:23
2
\$\begingroup\$

A couple of things i try to do first to debug it. For your ui just set the pixel shader to write out white with alpha set to 1.0f. If that is rendering and is no longer transparent then you can localize it. But if not i would also look at your blend state. I assume your ui isnt using a zbuffer? If it is then is it cleared every frame? You are rendering something so it's not around geometry.

I use a similar method to what you have done in the ui. I find just methododicly going through those items you usually find the issue. If not post again and i will try and think of other possibilities

\$\endgroup\$
1
\$\begingroup\$

In the event that somebody with a similar problem stumbles onto this thread, I'll share what eventually fixed my issue. I took ErnieDingo's advice and took a closer look at the z-buffer. Looking for things wrong with my z-buffer, I ran into this thread, which reminded me of a crucial detail I had forgotten.

In the code above, I'm using the same z-buffer for each canvas, which can be seen in the call to ClearDepthStencilView(DXEnv.depthBuffer, ...). Except, z-buffers can't be shared between render targets of different sizes, because the buffers are created with a specific size in mind. This was the monkey wrench in the machine, so I gave each canvas its own z-buffer and attached a unique depth stencil view to each z-buffer.

The new code for that for loop looks like this (I also went ahead and batched up my render-to-texture calls and my canvas renders, to avoid excessive switching between targets):

    for (int i = 0; i < canvases.Length; i++)
    {

        if (canvases[i] != null && canvases[i].active)
        {

            //set the new render target to be the canvas, and set texture to be the spritesheet
            DXEnv.context.ClearRenderTargetView(canvases[i].renderTarget, new Color4(1.0f, 0.0f, 0.0f, 1.0f));
            DXEnv.context.ClearDepthStencilView(canvases[i].depthView, D3D11.DepthStencilClearFlags.Depth, 1, 0);
            DXEnv.SetNewRenderTarget(canvases[i].renderTarget, canvases[i].depthView);

            //recursively render to the new texture
            textureUnlitShader.SetTextureResource(DXEnv.device, DXEnv.context, canvases[i].source);
            RenderPanelRecursive(canvases[i].root, matrices);
        }
    }

    DXEnv.SetNewRenderTarget(DXEnv.renderView, DXEnv.depthView);

    for (int i = 0; i < canvases.Length; i++)
    {
        if (canvases[i] != null && canvases[i].active)
        {
            canvases[i].CalculateModelMatrix(transforms[i]); //TODO: return types make this more intuitive
            matrices.modelMatrix = Matrix.Transpose(canvases[i].modelMatrix);

            //set texture to the new resource and reset render view, then render canvas
            textureUnlitShader.SetTextureResource(DXEnv.device, DXEnv.context, canvases[i].shaderTexture);
            textureUnlitShader.UpdateShader(DXEnv.device, DXEnv.context, matrices, canvases[i]);
        }
    }

NOTE: the above code doesn't showcase the fact that my differently-sized render targets also require different projection matrices to be rendered properly. If you have different sized render targets, each render target will also require its own localized projection matrix, just for the record.

Thanks for the help.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.