Know if you're fully utilizing the GPU

Question

I render ~17.000 VAOs each frame. 2.840.386 triangles. Only applying texture, nothing else.

I have three computers and the performance across them is not as expected.

• Cheap laptop(i3-4010U & Intel HD 4400) runs at ~20 fps.
• Desktop1(FX-9590 & GT 630) runs at ~65 fps.
• Desktop2(i7-5820k & GTX 970) runs at ~105 fps.

As the game is in a pretty early stage, the only CPU task each frame is to loop through a 3-Dimensional map containing VAO ids, and then render them.

When I look at the CPU Benchmarks and the GPU Benchmarks of my systems, I would expect a much bigger difference between my Desktop2 and the two other systems. The GTX 970's 3D score(8662) is more than 15 times better than the Intel HD 4400's (546), though it only run 5 times as fast. Even more ridiculous the GTX 970's 3D score is ~11 times better than the GT 630's (797), but run 1.6 times faster.

Even without taking the benchmarks into consideration I would still expect bigger difference. When I compare the performance differences in other games(Like GTA V) across my systems, there's a much bigger difference. Therefore I know the problem is my game, and not a hardware/driver related issue.

Am I right that I would expect larger differences between the different systems, if yes: How may I find the issue and how can I solve it?

Edit: I've read that GTX 590 is capable of 3.2 Billion triangles/second, where as my game is only able to squeeze ~300 million of a GTX 970.

Edit 2: I've tested my desktop1(GT 630) that runs 1080p at 65fps and if I change the resolution to 1 * 1, I only get a performance boost of 20fps(85fps).

Answer 1

TL;DR: Your rendering test is very simple with no post-processing so even an integrated GPU has an easy time with it.

---

GPU Benchmarks test many different aspects of a rendering pipeline and gives you back a single simplified number.

GPUs have multiple points that can bottleneck and final render time depends on the worst point of the pipeline.

Some examples of possible bottlenecks:

• Telling the GPU to start drawing (small number of primitives vs large batches)
• GPU rendering mode configuration time (to switch shaders, textures, etc.)
• GPU reading vertices from System RAM vs Video RAM
• Vertex format re-organizing/re-aligning due to GPU DMA limitations
• Vertex shader code processing (size + branches + # of cores)
• pixel shader code processing (size + branches + # of cores)
• texture fetches (amount, size, dependent reads)
• post-processing steps (FBO flush -> texture -> pixel shader -> FBO)
• Memory bandwidth
• cache sizes & organization
• bus bandwidth between CPU<-->GPU (in some rare extreme cases integrated GFX can be faster since they share the same cache & memory)
• Triangle to tile draw combining
• fixed point vs floating point shader performance

GPUs can "faceplant" in terms of performance past a certain point on any of those items.

For example, if GPU-Model-A has 8KB of instruction cache for each execution units but runs slightly faster than GPU-Model-B that has 16KB.

Model-A takes 7usec to process a 6KB shader while Model-B that takes 9usec. But if you're using a 12KB shader, Model-B can fit the entire shader it its cache but Model-A has to run it in 2 cache loads, which adds 30usec* to Model-A's execution, suddenly Model-B runs much faster in comparison.

*These are made-up numbers, actual numbers vary.

So the artificial benchmark final number depend on how much weight they assign to the score for each part they test and how much load and what type of load they put on the GPU & CPU.

---

In order to figure out which part is the bottleneck you must go through an elimination process where you lighten up the load off of each items and check the performance difference.

For example:

• reducing screen resolution
• disabling post effects
• reducing texture sizes
• reducing shader complexity (while still reading the textures)
• removing dependent texture lookups (while still reading the textures)
• rendering larger batches of triangles to reduce draw calls
• disabling skinning

And check the performance gain of each single item or combination.

Answer 2

Note: While FPS can be safely used to compare values, for measuring differences(for optimizations) you should measure frame times.

There are programs that help you tell whether your application is cpu or gpu limited. One of them is AMD GPU PerfStudio, which as the name suggests is vendor-specific. I am sure NVidia has a similar program too, but I don't know its name. EDIT: I just remembered a basic GPU utilization meter is usually included in overclocking tools and and in the driver settings.

Also you should make sure that the cards are doing the same thing, there are often vendor-specific optimizations or effects that are tuned in relation to the GPU you have, these are usually set in the GPUs control panel. And make sure that the resolutions are the same too as laptops usually have lower resolution and this may strongly affect performance. The true performance of a high-end GPU may only be visible on higher resolutions.

Also bear in mind that the additional speed of a more advanced GPU may only kick in when you are doing more stuff, using more memory, and using more advanced features, which you said you aren't. GPU benchmarks are usually built to attempt to use all features and evenly distribute the work between all GPU blocks.

We have no way to know if the benchmarks points scale linearly with performance. The benchmarks may also be tuned to use tessellation and other new features which only recently been optimized in hardware, which will bring new-gen GPUs up-front, while in a simpler setting their power may be almost the same.