4
\$\begingroup\$

I've looked around on the web trying to find a good answer to this, but am unable to find one. Different forums mention driver bugs and the likes, but these discussions are usually almost 10 years old, so bugs should probably have been fixed by now. So, to the question!

I'm drawing a mesh that consists of 628 008 vertices, normals and texture coordinates. I also have indices for the mesh, but the indices are the worst kind of indices: they do not re-use vertices, but actually just list the vertices in order (or close to). So we have 628 008 indices as well. Yes, I can re-write this to re-use vertices, but that's not really where my question is going.

The mesh is drawn using VBOs (vertices, normals, texture coords) in a VAO, and two different shaders: one for texturing one part of the mesh, and another shader for another non-textured part. To do this, I'm using the OBJ format and material groups (exported from Blender). The mesh is split into material groups, describing with which material a batch of indices shall be drawn.

The VBOs are set up to be drawn as GL_STATIC_DRAW since it is never changed.

I then loop over the material groups and draw the range of indices described in each group, like:

  for( const auto& group : mesh)
    if( group should use texturing )
    {
      enable texturing shader with phong material
      set texturing uniforms
      set material stuff
      set mvp, etc
    }
    else
    {
      enable phong material shader
      set material stuff
      set mvp, etc
    }
    draw(group)
  }

There are 13 different groups, so the draw command will be called 13 times.

I've tried drawing the mesh in five different ways and the results are quite surprising.

  1. glDrawArrays.
  2. glDrawElements with indices fed using pointer to index array. (reffered to as CPU-side indices below)
  3. glDrawElements using GL_ELEMENT_ARRAY_BUFFER and offset fed to command
  4. glDrawRangeElements with indices fed using pointer to index array.
  5. glDrawRangeElements using GL_ELEMENT_ARRAY_BUFFER and offset fed to command

The [start, end] vertex indices fed to glDrawRangeElements are calculated when reading the mesh at start-up so they are correct and precise. This could be a performance problem otherwise, according to the spec.

The results in ms (measured on CPU side using C++11 timers, using 500 samples and calculated mean value, collecting 3 values since the first frame's timing is huge):

glDrawArrays:
            Median: 8.0 Mean: 8.9 Max: 217.0 Total: 4285.0 Samples: 482
            Median: 8.0 Mean: 8.8 Max: 43.0 Total: 4345.0 Samples: 494
            Median: 8.0 Mean: 8.2 Max: 14.0 Total: 4125.0 Samples: 501

glDrawElements:
GPU Indices (ELEMENT_ARRAY): 
            Median: 34.0 Mean: 44.1 Max: 597.0 Total: 20388.0 Samples: 462
            Median: 34.0 Mean: 33.9 Max: 63.0 Total: 16973.0 Samples: 500
            Median: 33.0 Mean: 33.5 Max: 80.0 Total: 16797.0 Samples: 501
CPU Indices (pointer to CPU-side indices):
            Median: 9.0 Mean: 10.3 Max: 236.0 Total: 5027.0 Samples: 488
            Median: 9.0 Mean: 9.2 Max: 32.0 Total: 4581.0 Samples: 500

glDrawRangeElements:
CPU Indices (pointer to CPU-side indices):
            Median: 9.0 Mean: 10.1 Max: 221.0 Total: 4640.0 Samples: 459
            Median: 10.0 Mean: 9.6 Max: 22.0 Total: 4780.0 Samples: 500
            Median: 10.0 Mean: 9.5 Max: 32.0 Total: 4774.0 Samples: 501

GPU Indices (ELEMENT_ARRAY): 
            Median: 35.0 Mean: 36.9 Max: 597.0 Total: 16101.0 Samples: 436
            Median: 34.0 Mean: 35.0 Max: 55.0 Total: 17517.0 Samples: 500
            Median: 39.0 Mean: 39.1 Max: 64.0 Total: 19448.0 Samples: 497

So my huge surprise is: How come GPU-side indices takes so much longer? The frame rate literally drops to less than half (from about 34 fps to 15 fps).

What can be the reason for this? My guess would have been that the performance timings should be in this order (since we have crappy indices and they aren't really re-using anything):

  1. glDrawArrays <- Fastest
  2. glDrawRangeElements (using ELEMENT_ARRAY)
  3. glDrawElements (using ELEMENT_ARRAY)
  4. glDrawRangeElements (using pointer to indices)
  5. glDrawElements (using pointer to indices) <- Slowest

Why do you think that the performance is so bad using ELEMENT_ARRAY_BUFFER bound to the VAO? Is this driver specific? Is it because there are so many indices and memory have to be swapped around in the driver?

Hardware: MacPro running Windows 8 Dual AMD FirePro D300, 2048 MB dedicated graphics memory

\$\endgroup\$
6
\$\begingroup\$

I would expect glDrawArrays to be fastest on both the GPU and CPU for this scenario, primarily because you're saving the GPU memory bandwidth otherwise used for fetching the index data and the CPU cost of managing the index buffer.

I do not expect glDrawRangeElements to be any faster than glDrawElements - in fact, I would expect it to be slightly slower. The range passed to the function is intended to be used to allow a driver to know which sections of the vertex data need to be made available to the GPU. However, if you're using a vertex buffer, this is not useful information. Further, it's also just a hint - drivers are expected (although technically not required) to render correctly even if the range is wrong, and it's surprising how many applications do get it wrong. So, we generally ignore those parameters.

As for why glDrawElements is faster with client index data, that's odd, but really depends on the sequence of calls. You say your overall mesh has 600K+ vertices, but it's broken into groups. What is the size of each group? Each time you switch states (such as binding a new index buffer), the driver needs to validate the contents of the buffer, which takes time. If you're rapidly switching between two states, then this could be where your CPU time is going. When you use an index buffer, each switch is (potentially) a new buffer and is more work for the driver to validate. If you use client indices, even though the driver might be transferring indices to the GPU, it's all technically the same buffer - a gigantic buffer that starts at CPU virtual address 0.

Of course, it's also possible that the driver doing something silly.

Ultimately, 40 FPS is pretty low for drawing 600K vertices. Something bad is happening.

Here are a few things you can try to improve performance:

1) If your fragment shader and fixed function state is basically order independent (depth test on, stencil and blend configured for order independence), sort the groups in the mesh by textured vs. untextured, then draw all of one set followed by all of the other with no state changes in between. You can even sort further - by texture or other parameters, for example.

2) If any state is common between groups (mvp matrix, for example), make sure you're not setting it redundantly.

3) Make sure all groups in the mesh are using the same vertex and index buffers, ideally bound into the same VAO. When you draw, you can use the {first} or {baseVertex} parameters to offset into a larger vertex buffer. You can even use the same buffer object for both index and vertex data - just subdivide it.

4) As a last resort, you could try making all groups use the same shader by simply binding a dummy texture with an opaque white pixel into it for the untextured groups. If you're really CPU bound here, the extra GPU cost of fetching the (unneeded) texture data and a few extra shader instructions probably won't matter.

Once you do get the glDrawElements path going as fast or faster then the glDrawArrays path, do consider optimizing the vertices for reuse. A triangle soup mesh with perfect reuse can perform up to three times better than the same mesh rendered with no reuse.

\$\endgroup\$
  • \$\begingroup\$ I'll try to comment/answer on some of your questions. 1. I read somewhere on stackexchange that the min/max values might have to be calculated for glDrawElements, and that glDrawRangedElements would be faster because of that. But if it is like you say (and I trust that it is) that the driver won't trust the range, it's redundant anyway. Perhaps it should be deprecated in the next OpenGL then (the less, the merrier...) 2. I'm also baffled by this. The groups are about 60k vertices each. Say 4 are twice the size (120) and 9 are smaller or about 60k. I'm only using 1 index buffer for all groups. \$\endgroup\$ – AzP Feb 19 '16 at 10:37
  • \$\begingroup\$ (contd.) So I'm always supplying an offset and count to the same index buffer, regardless of using glDrawArrays/Elements/RangedElements. \$\endgroup\$ – AzP Feb 19 '16 at 10:38
  • \$\begingroup\$ Btw. the 40 fps is for rendering a "normal" frame in our application, so I'm rendering some other stuff as well. But the mesh is the largest thing. We also have event based rendering (not render loop), so I have to move the mouse around in the view to actually force it to render, even though I easily get it to 60 fps if the rendering speed allows it. It's a medical software with several rendering views (and contexts). \$\endgroup\$ – AzP Feb 19 '16 at 10:42
  • \$\begingroup\$ Thank you for the tips, I'll try them out! I was also planning on creating color VBOs (and interleave vertex/normal/texture/color) instead of sending them as uniforms, and setting texture ID as alpha value (since nothing is transparent). But it feels wasteful to send a color value for each vertex when there are really only 3 different colors used, any way to do it in OpenGL without having to set uniforms? \$\endgroup\$ – AzP Feb 19 '16 at 10:45
  • 2
    \$\begingroup\$ I have found out about the horrible performance impact! We are using GL_DOUBLEs everywhere because we usually receive our data (from physics calculations, etc) so I was drawing this mesh in the same way without thinking about it. When switching it to FL_FLOAT, the time reduced to Median: 0.0 ms Mean: 0.0, for glDrawArrays and Median: 1.0 Mean: 0.5, for glDrawElements. The performance difference is completely insane. \$\endgroup\$ – AzP Mar 2 '16 at 9:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.