What calls trigger a new batch?

Question

I am finding my project is starting to show performance degradation and I need to optimize it. The answer to my previous question and this presentation from NVidia have helped greatly in understanding the performance characteristics of code using the GPU but there are a couple of things that aren't clear that I need to know to optimize my drawing.

Specifically, what calls make the distinction between batches.

I know that any state changes cause a new batch, so that includes:

• Render State Changes
• Buffer Changes
• Shader Changes
• Render Target Changes

Correct? What else counts as a 'state change'?

Does each DrawPrimitive() call constitute a new batch?** Even if I were to issue the same call twice, with no state changes, or call it once on on part of the buffer, then again on another?

If I were to update a buffer, but not change the bindings, would that be a new batch?

That presentation and a DX9 page suggest using all of the texture slots available, which I take to mean loading multiple objects in 'parallel' by mapping their buffers/shaders/textures to slots 1-16. But I am not sure how this works - surely to do this you would need to change the buffer binding and that would count as a state change? (or is it a case of you do but it saves 16 calls so its OK?)

Answer 1

You can say that any change to a data that will be processed by a subsequent call to Draw**Primitive() will trigger a state change.

If you call a Draw two times without changing anything, it's likely that your second call won't any anything from the first (well, except if you're drawing with alpha).

Change a constant buffer will trigger a new batch for sure.

With DX9 (and all 3D API in general) the state change that costs the most is the Texture slots one, by using all the 16 slots available you intent to minimize the texture switches. A good thing to do is (when possible) rendering the primitives using the same textures in consecutive calls.

It's not easy to figure things out in DX9 the API design doesn't help much to understand how to do things efficiently. DX10/11 APIs are really an improvement in this matter as everything is regrouped/organized into blocks where the creation and applying steps are clearly differentiated.

GPU optimization all in all is about:

1. Reducing the number of call/changes you make.
2. Make sure you disrupt the rendering flow as less as possible (and any change to a buffer that will be immediately used is a notable disruption).

Bonus answer: if you want to do Object Picking, with not going with a CPU based ray-tri intersection routine? It's pretty fast if you early reject objects with a first ray-bounding box intersection.

Answer 2

Back to back draw calls are extremely efficient. It should perform nearly identically to 'hardware instancing' features available in the APIs, aside from the added overhead of calling the API repeatedly. This is how we render relevant on-screen terrain fragments as well as all the random weapons in Borderlands.

You can usually consider updating resources (texture,constants,vertex,index) a separate issue to changing state. These involve additional penalties (lock stalls especially in SLI setups) depending on the driver. A lot of these are hidden by well written drivers by internally managing their own staging resources that get DMA'd asyncronously and freed behind the scenes.

Changing which resource objects (including rendertargets) the GPU is using is relatively cheap. The cost is primarily in the fact the GPU caches have to be cleared when they are changed, but it is normally negligable. The major state blocks (depth/stencil, blend/colorwritemask, rasterizer/scissor) historically are a bit trickier since a good number of those states depend heavily on other states, and is one of many reasons why older APIs like D3D9 are lazy about doing state work until draw calls occur. For example you might have the depth test set to less-equal but no depth buffer bound, but the driver has to turn it off when this happens.

If you are working with an older api like D3D9 one of the biggest free boosts you can get is to batch up all the shader constants into a giant struct and only upload the relevant dirty region before you call draw, as calling into the SetVertexShaderConstantF over and over is pretty expensive.