why is it faster to draw lots of small arrays than one big array?

Question

This application on-line here has interesting performance characteristics:

Sorting the rectangles to be drawn so it keeps having to change colour is faster than sorting the rectangles by their colour and drawing all of each colour at once...

(Timings are printed to Javascript console. Javascript profiling and understanding what is happening between the emitting of draw ops and the actual rendering of frames is, well, guesswork and disappointment. But the result is still surprising (to me).)

I'm very interested in the fps logged to the javascript console when run in your browser, and how it differs when sorted by colour vs sorted by z

Now naturally I assume GPU batches are expensive and in my normal apps I get that impression, but I cannot distill a simple test case to show this; quite the opposite, in fact!

Deep in my code, it places all the vertex data interleaved into a single big VBO array. It then calls glDrawArrays as often as needed. So drawing a rectangle (6 vertices, each 4 floats) in three colours can be:

• Red*1,Blue*1,Red*1 = 3 glDrawArrays calls
• Red*2,Blue*1 = 2 glDrawArrays calls, but is slower

I am not sorting, creating, uploading the vertices each frame. I compute the buffers to draw once. Each frame, I simply call glDrawArrays as often as needed.

Because I've built this vertex attribute VBO once, and keep reusing it, I have a tight little draw-loop for rendering each frame. So the Javascript code is doing more work when I have lots of little calls, and less work when I'm emitting concatenated rectangles.

So why is it faster to call glDrawArrays multiple times with small sub-sequences of the array, instead of calling it only a few times and telling it draw long sequences from the array?

Related: there's +500 pts up for grabs over on StackOverflow for the best general discussion about optimising the drawing of a 2D scene.

Answer 1

As far as I can tell from my working knowledge of OpenGL, the act of sorting all of your rectangles into one big draw call means the CPU is wasting the first half of each update loop carefully and neatly organizing everything while the GPU twiddles its thumbs waiting for the draw call to come in.

Conversely, when when you tell each object to draw individually one after another, the GPU doesn't have to wait around, it's always working at the same time. This way you draw in less than half the time, firstly because the CPU doesn't have to waste time organizing everything into one big draw call every frame, and secondly because the GPU doesn't have to wait for the CPU to completely finish before it can start rendering.

Now this may be a little different in web applications, but most drawing processes generally work the same.

Answer 2

What you describe is not sprite batching. Sprite batching is sorting so that the GPU doesn't have to change textures too often (expensive because textures have to be loaded from GPU memory).

Colour data is usually stored in the vertex attributes. If two objects have different colours this doesn't matter, since we always have to read the colour data and 'changing colours' does not trigger loading of any resources.

To give a simpler comparison. If you have to add up five numbers to each other, it won't help first sorting the numbers before you sum them up. The number of instructions stays the same and sorting adds overhead.

Answer 3

Just to elaborate on Stomy's answer - I'd hesitate before describing time spent sorting as "waste"; if the performance gain from being able to batch outweighs the performance loss from sorting, then it's perfectly valid and a reasonable use of CPU time.

There's also a misunderstanding in Stomy's answer whereby the CPU and GPU actually don't operate in a synchronous order as he describes; instead the CPU will submit commands to a driver-managed command buffer and return immediately, which the GPU will then come along and pickup some time later, so they're quite asynchronous.

The trick is to be able to balance your workload between CPU and GPU so that neither ever gets to a situation where it needs to wait on the other before it can continue.

That hints at what's going on here - you seem to be spending a lot of CPU time on sorting and you're not gaining the desired performance from it. I don't know what you're using for your sorting, but one would expect that with JavaScript being a generally slower language, it may not be the most optimal for any kind of more CPU-intensive work such as this; your sort algorithm may also need looking at (e.g. you may be using bubble sort).