I'm very new to graphics programming, and as I understand it vertex shaders are called per vertex and fragment shaders are per pixel (ignoring anti-aliasing). When it comes to optimization all sources I refer to say that calculations that could be done in either shaders should be done in the vertex shader, and it makes sense: 5*8 done 3 times is way better than 5*8 done 300'000 (640×480 for example).

And that's fine for this scale but the question almost poses itself: what happens when the amount of vertices exceeds the resolution. Here's a practical example: a typical AAA game could have upto 3'000'000 polys rendering on the screen, for the sake of simplicity let's say it's a very detailed triangle strip, that would give it exactly 3'000'002 vertices; while a full HD screen has 1920*1080 ≈ 2'000'000 pixels. As you can tell a difference of a million operation could be a substantial performance increase.

Is my thought process sound, or does my ignorance of some details leave something to be desired?


1 Answer 1


The calculation falls apart like so:

If every one of those 3 million polygons rasterizes to at least one pixel on average, then doing the operation in the fragment shader still results in it being evaluated (up to) 3 million times, even if your screen or output buffer has only 2 million pixels. If each one rasterizes to two pixels on average, then you might get up to 6 million fragment shader invocations - the screen size doesn't actually cap the number of times you perform this expensive operation.

The reason is overdraw. Your graphics card isn't guaranteed to draw just the frontmost polygon to each pixel. If the objects are drawn out of front-to-back order (say, terrain drawn before a dynamic object sitting on top of the terrain), or if the triangles within a single object aren't ordered front-to-back, or if you have intersecting polygons, or if you're rendering with transparency, you can potentially write to each pixel in the output many times over the course of rendering.

The amount of overdraw you get depends a lot on the contents of the scene and the viewpoint, so it can vary wildly from frame to frame. So if there are expensive operations in the fragment shader that could be in the vertex shader instead, this can contribute to performance-sapping spikes, compared to somewhat more predictable/steady/controllable vertex counts.

For many effects we can mitigate the overdraw problem with various forms of multi-pass rendering: first we render the geometry of the scene using a reasonably cheap shader, paying some overdraw cost where it overlaps itself, but we can accept this cost because each invocation of the fragment shader is inexpensive (usually just writing out some values to a G-buffer). Then we follow up with a screenspace quad that reads this already-rasterized scene information and renders a more complex effect on top of it. Since this is a single layer of geometry rendered in screenspace, we can guarantee it runs only once for each pixel it covers, not once for every layer of geometry that touches that pixel.

This is the basic idea behind Z-prepass & deferred lighting, or screenspace effects like SSAO & post-processing filters.

Looking at it another way though: if you have a triangle that rasterizes to only a few pixels, wouldn't you rather not draw such a triangle at all? It's probably ripe for level-of-detail simplification into a coarser mesh or imposter, or culling due to distance. So if you have so many such tiny triangles that moving operations to the fragment shader seems like a savings, you'll probably get bigger wins by investing in LoD & culling systems to draw fewer tiny triangles.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .