OpenGL memory optimization and batching technique pitfalls

Question

I'm studying OpenGL optimization techniques. What I'd like to achieve is to emulate as closely as possible Mantle/DirectX12 programming patterns (aggressive batching, flexible memory handling, complex pipelines) with current OpenGL API. The assumption I'm making here is that hardware behavior is almost the same, with the OpenGL layer being a bit restrictive and not matching accurately the underlying architecture.

I'm considering the optimization technique proposed here:

http://www.openglsuperbible.com/2013/10/16/the-road-to-one-million-draws/

The article suggests allocating a giant buffer and writing a custom allocator to fill it with vertex, instance and texture data, then drawing it with glMultiDrawElementsIndirect. Immutable storage, triple buffering and persistent mapping can also be used for best streaming performance (see http://www.bfilipek.com/2015/01/persistent-mapped-buffers-in-opengl.html).

That seems pretty close to my goal, however I have some doubts:

1. I read that it's better not to overuse GPU memory on PC because it's also needed by other applications. However, when using this technique with dynamically loaded, continuously changing scenes (e.g. very large tiled worlds) we end up using much more memory than actually needed because we can't predict how much data we'll need to store. Is this a real concern? If yes, how can I address the problem?

2. View frustum and occlusion culling force you to continuously rewrite instancing and command data. Is this really good? Does decreased CPU load reward time spent copying around memory chunks? Does memory bandwidth significantly impact performance in this case?

3. Is the "one shader for all materials" method based on subroutines more efficient (higher frame rate with same input data) than modifying the program pipeline? What's the cost of binding a different pipeline or switching a shader in the current pipeline?

Answer 1

However, when using this technique with dynamically loaded, continuously changing scenes (e.g. very large tiled worlds)

You wouldn't use this for the level geometry.

Your chunks should map in their own static buffers on demand, just like you normally would.

View frustum and occlusion culling force you to continuously rewrite instancing and command data. Is this really good?

You have to constantly rewrite that data anyway for your dynamic objects, so you're hardly losing anything with this technique.

The streaming persistent buffers remove latency; they don't add additional copying. Remember that the driver is allocating GPU memory repeatedly for buffers that you are discarding (i.e. the old recommended practice) so there's already a lot more copying going on than might be apparent. The persistent streaming buffers just put you in control of that rather than making it magical driver shenanigans.

Is the "one shader for all materials" method based on subroutines more efficient (higher frame rate with same input data) than modifying the program pipeline? What's the cost of binding a different pipeline or switching a shader in the current pipeline?

If you're just targeting current/modern hardware (with latest drivers), subroutines should be faster than swapping shader programs. A subroutine should be implemented as a simple indirection in the shader machine code. You want the subroutine selection to be as uniform as possible to ensure maximum GPU shader core throughput; same as with any uber-shader approach, really, except that subroutines will be faster than branching (usually).

Switching shader programs involves various pipeline state changes, which can be complex commands. The cost is not fixed by the spec so it of course will vary by hardware. A single GPU shader cluster can only run one program at a time, so on "old" hardware a shader change can involve an entire pipeline stall, while recent hardware can often run a handful of different shaders at the same time (though the driver may not allow a single application to take advantage of this).

Profile your specific game and specific content on target hardware to find out for sure.