Frame graph in Vulkan: from theory to implementation

Question

I've watched the excellent talk from Yuriy[1] and read a blog post[2] multiple times already, but I still cannot find the answers to the questions I have.

I really like the idea of defining a graph of "virtual resources", it gives a lot of flexibility, maintanability, and ease of use (think about synchronization) in the long term. First, I thought that the easier way to solve my problem (described below) is to implement this system so that the graph building and the transformation to the actual graphics API wouldn't be done every frame, only when the conditions (eg. graphics settings) change.

However, extending the "only when the conditions change" statement to include game logic-related decisions, we get to a point where the frame graph must be rebuilt almost every frame. This is also what Yuriy suggested in his talk and I agree with it.

The main problem

In Vulkan the whole pipeline (with the corresponding render pass) must be pre-built. So the rebuilding of the whole graph every frame can be a bottleneck as I can imagine. The pipeline cache could speed up things (in theory) but avoiding sudden frame-rate drops is top priority. However, if a pipeline (especially the shaders) have not yet been built, it could cause a hitch up. And pre-warming the cache(s) with every possible permutations defeat the purpose of the whole system.

The question

So I'm wondering if there is a way to keep the best from both worlds, rebuilding the whole graph every frame while avoiding sudden frame drops from pipeline creation.

If there is no such solution, what would be the closest thing to this kind of dynamic whole frame predefinition? Should I fall back to the rarely rebuild graph, and use another method to skip (sub)passes?

References

[1] FrameGraph: Extensible Rendering Architecture in Frostbite (GDC Vault)

[2] Render graphs and Vulkan — a deep dive (link)

Answer 1

Because i saw this post while wondering it myself i wanted to give an answer on how i implemented it, maybe it will be a use to someone.

As you said yourself pipeline creation every frame doesn't make sense as we are building new render passes each frame, and also just rebuilding everything each frame sound like to much wasted work. One solution is of course to use the new VK_KHR_dynamic_rendering implemented in Vulkan 1.3 making it easier. Another would probably be to create render passes that are compatible with the ones we will use later in the frame so we can build pipelines out of them.

The third solution i could think of and the one i liked the most is to just build all passes and attachment details at the beginning of the application. With it we know all the passes and can build our pipelines and materials when ever we desire during the applications lifetime. It also goes well with the material system i saw here: https://technology.riotgames.com/news/better-living-through-materials. Where passes could be in this case techniques.

With it we can also cache frame buffers and image/imageviews if we want. In my case i have two of each for each frame-in-flight. Doing this we have done the setup and half of the compile phase as proposed by Yuriy.

The other half of the compile phase and the execute can still be done each frame. The idea here was to still make it necessary to activate each frame what techniques will be used. And with the activation we pass the execute function that will be called.

For example frameGraph.Activate("Shadowpass",[=](FrameInputs& inputs){//Do stuff//}); You could like in my case also pass a pointer to an object implementing an execute interface.

And then do the second part of compiling the graph so to speak where it will check if these activations are possible given all the techniques and how they relate to each other. This makes branching possible like shown in the gdc talk.

if(Debug){
  frameGraph.Activate("Debug View",[=](FrameInputs& inputs){//Do stuff//});}
else{
  frameGraph.Activate("Lighting",[=](FrameInputs& inputs){//Do stuff//});}
}

After that just run the frame graph and it handles everything else like calling all the activated passes and passing the proper inputs to them.