d3d11 renderqueue multi pass rendering design best practice

Question

The goal: issue least calls to d3d11 ImmediateContext and render advanced effects like toon / outline / anything that requires rendering the same vertex buffers with at least two different vertex shader and pixel shader programs efficiently by sorting opaque drawcalls front to back and transparent drawcalls back to front.

What I did: Abstract calls to d3d11 immediate context by a DrawCall struct that acts as a gpu command recorder. A call to DrawCall.SetIndexBuffer will create a c# Action that will issue a call on the ImmediateContext upon invoke call. The DrawCall internal gpu command list is sorted the following way: set shaders, set constantbuffers, set blendstate, set stencilstate, set vertexbuffers.... I am using sharpdx and c#.

My problem: I use a 64bit sortkey for my gpu commands. When rendering complex effects like outline or anything that requires state changes to outputmerger the whole idea with sorting doesn't seem right... example: I render opaque objects first and then transparent objects... I want to render an object opaque with a diffuse shader and it should have a red transparent outline. For this effect I have to modify stencil and blend states in between drawindexed calls. According to my sortkey those gpu commands are on different draw loops opaque<>transparent but the stencilstate requires both drawindexed calls to be one after another.

Should I implement a chained gpu command for multipass rendering or are there better ways to handle such effects?

Answer 1

I want to post an answer. I decided to go for a flexible solution to achieve the following features:

• Rendering to different layers (world, overlay text, gui, etc.)
• Multipass rendering (render same submesh with two different sets of vertex and fragment shaders, aka materials)
• Additive blending aka transparent rendering
• using stencilbuffer inside material passes as a special command
• shared vertex buffers, shaderresourceview (aka texture)
• dynamic buffer updates

Thats how I implemented my flexible render pipeline:

1. Traverse my scenegraph and look for objects that have a transform, material and mesh component
2. when all three components are found on an object I ask the renderqueue to create a drawcall for me. The creation must be inside the queue to prevent reusing the same drawcall multiple times.
3. a drawcall is basically a sorted collection of 'commands' (see command pattern) that run on the d3d11 ImmediateContext at some later point. I first collect the drawcalls (bundles of commands) and then later i sort the drawcalls according to renderlayer, blendstate and material pass Index
4. I ask every component (transform, material, mesh) to contribute to building the drawcall by populating the commands inside the sorted collection. every command has a unique key to prevent adding the same command twice inside a drawcall. I basically just ask the transform to give me a constantbuffer of world matrix, the material to set correct shaderprograms and textures AND decide the draw method (Indexed, instances, whatever), the mesh to set correct vertex buffer and indexbuffer. A mesh might consist of many submeshes. so for each submesh a corresponding material with atleast one material pass must exist in order to render the submesh.
5. now that i have a collection of drawcalls I start to sort the drawcalls. By sorting i mean put fractions of drawcalls into some collections. Thats the sorting order: sort by renderlayer (world, overlay, gui), sort by render order (first opaque, then transparent. both have a specific range of int, like in unity game engine), sort by pass index (an object can be drawn opaque with an transparent outline)
   6. invoke the commands. call ImmediateContext.Flush at the end to actually run all invoked commands, because I render to an offscreen rendertarget and not the backbuffer directly.

So far it works great and I can make quiet some effects with it. For my intended use I only use immediate rendering with basic lighting and no multithreaded drawcall collecting, because my scenes are well below 2k objects. Another bonus I do is sort my drawcalls also by distance. for opaque objects I use front to back sorting by camera distance to reduce overdraw aka running the fragment shader unneccessary for objects that will be overdrawn later. For transparent objects I sort back to front to achieve maximum overdraw, otherwise transparency with additive blending doesnt work. there is a good nvidia article on blending and in khronos group wiki also (tldr: transparency cant be achieved exact in realtime, thats why we do it approx by sorting objects back to front).

Answer 2

Background: There is no short answer here, but as someone who also codes in C# and Sharpdx, I can go over some of the learnings I have.

I sacrificed some of my flexibility for specific for fixed function draw actions. That being for example, draw my terrain, draw my models, text etc. But, what I do differently is that there is a pass specific to shadows, a pass specific to above water, below water, and transparency. I tag dynamic assets to render in these passes, so that I have the flexibility to select what rendering needs to be applied.

What I end up having is a fixed graphics pipeline with components within it that allow for flexibility, it also made the job of using deferred rendering much easier.

I, like yourself started out with a fully fledged flexible pipeline, but ended with a more fixed rendering pipeline wtih some flexibility. This is where I think you will end up, you have a need to control some elements/anchor points of your pipeline which allowing for your engine to take care of some of the finer details. As from your problem, you want to exert some specific control of the drawing at times.

On a performance note, you really don't want to be changing the state too often because you flush the GPU pipeline. On more modern GPUs, this isnt as bad (usually only affects a few cycles). So you may end up grouping your drawing commands differently, that is draw all relevant objects with your diffuse shader, then modify stencils states etc then draw all the objects again etc...

Chaining GPU commands (say through DrawIndirect) also will give you a performance improvement if your GPU is being starved by the CPU and you want to feed the pipeline quickly. I tested for example deferred rendering in my game engine which allowed for the draw commands to be recorded and buffered. This had significant performance improvement in Debug but literally no frame rate improvement in release.

Gl, hope you find your solution.