How to cache resources in my homebrew rendering system

Question

Background:

I am designing a simple 3D render system for an entity component system type architecture using C++ and OpenGL. The system consists of a renderer and a scene graph. When I finish the first iteration of the renderer, I might distribute the scene graph into the ECS architecture. For now it is placeholder one way or another. If possible, the following are my goals for the renderer:

1. Simplicity. This is for a research project and I want to be able to easily change and expand my systems (hence the ECS approach).
2. Performance. My scene might have many small models and also large volumes with a lot of geometry. It is not acceptable to acquire objects from the OGL context and buffer geometry every render frame. I'm aiming for data locality to avoid cache misses.
3. Flexibility. It must be able to render sprites, models, and volumes (voxels).
4. Decoupled. The scene graph may be refactored into the core ECS architecture after I write my renderer.
5. Modular. It would be nice to be able to swap in different renderers without changing my scene graph.
6. Referential transparency, meaning that at any point in time I can give it any valid scene and it will always render the same image for that scene. This goal in particular is not necessarily required. I thought it would help simplify scene serialization (I will need to be able to save and load scenes) and give me flexibility to swap in different scenes during runtime for testing/experimentation purposes.

Problem and ideas:

I have come up with some different approaches to try but I'm struggling with how to cache the OGL resources (VAO, VBOs, shaders, etc) for each render node. The following are the different caching concepts I've thought up so far:

1. Centralized cache. Each scene node has an ID and the renderer has a cache that maps IDs to render nodes. Each render node contains the VAO and VBOs associated with the geometry. A cache miss acquires resources and maps the geometry to a render node in the cache. When the geometry is changed, a dirty flag is set. If the renderer sees a dirty geometry flag while iterating through the scene nodes, it rebuffers the data using the render node. When a scene node is removed, an event is broadcasted and the renderer removes the associated render node from the cache while releasing resources. Alternatively, the node is marked for removal and the renderer is responsible for removing it. I think this approach most closely achieves goal 6 while also considering 4 and 5. 2 suffers from the extra complexity and loss of data locality with map lookups instead of array access. 3 suffers from the homogeneous cache.
2. Distributed cache. Similar above except each scene node has a render node. This bypasses the map lookup. To address data locality, the render nodes could be stored in the renderer. Then the scene nodes could instead have pointers to render nodes and the renderer sets the pointer on a cache miss. I think this kind of mimics an entity component approach, so it would be consistent with the rest of the architecture. The problem here is that now scene nodes hold renderer-implementation-specific data. If I change how things are rendered in the renderer (like rendering sprites vs volumes) I now need to change the render node or add more "components" to the scene node (which means changing the scene graph as well). On the plus side, this seems like the simplest way to get my first-iteration renderer up and running.
3. Distributed metadata. A renderer cache metadata component is stored in each scene node. This data is not implementation-specific but rather holds an ID, type, and any other relevant data needed by the cache. Then cache look-up can be done directly in an array using the ID, and the type can indicate which type of rendering approach to use (like sprites vs volumes).
4. Visitor + distributed mapping. The renderer is a visitor and scene nodes are elements in the visitor pattern. Each scene node holds a cache key (like the metadata but just an ID) that only the renderer manipulates. The ID can be used for array instead of generalized map look-up. The renderer can allow the scene node to dispatch a different rendering function based on the scene node's type, and the ID can be used by any cache. A default or out-of-range ID would indicate a cache miss.

How would you solve this problem? Or do you have any suggestions? Thanks for reading my wall of text!

Answer 1

After re-reading your question, I strongly feel you are overcomplicating the problem. Here's why:

1. There are only really two types of rendering system: Forward, and Deferred, none of which are dependant on a scene graph.

2. The only performance issues you should really get with any rendering system, should come from high poly count, and inefficient shader and client code.

3. Cache misses do indeed reduce performance, but they aren't quite the monsters you may think they are. 80% of your performance improvements will be from a more efficient algorithm. Don't make the mistake of pre-optimising your code.

That said:

If you are using a homebrew scenegraph then you should already be using a "Renderer" interface (or base class) to design the rendering part of your scenegraph code. Visitor pattern using double dispatch is a good approach to this, as you may well be using many types of graph nodes such as colour, texture, mesh, transform etc. In this way, during the render cycle, all the renderer has to do is walk the scene tree structure depth-first, passing itself as an argument. In this way the renderer is basically just a collection of shaders and maybe a framebuffer or two. The result of this is that search/removal code is no longer necessary for the rendering system, just the scenegraph itself.

There are certainly other ways to tackle the problems you face, but I don't want to give too long winded an answer. So, my best advice is get something simple working first, then expand on it to find it's weaknesses, then experiment with other approaches and see where their strengths/weakness lie in practice.

Then you will be well placed to make an informed decision.