Suitability of ground fog using layered alpha quads?

Question

A layered approach would use a series of massive alpha-textured quads arranged parallel to the ground, intersecting all intervening terrain geometry, to provide the illusion of ground fog quite effectively from high up, looking down, and somewhat less effectively when inside the fog and looking toward the horizon (see image below).

Alternatively, a primarily shader-based approach would instead calculate density as function of view distance into the ground fog substrate, and output the fragment value based on that.

Without having to performance-test each approach myself, I would like first to hear others' experiences (not speculation!) on what sort of performance impact the layered alpha texture approach is likely to have. I ask specifically due to the oft-cited impacts of overdraw (not sure how fill-rate bound your average desktop system is). A list of games using this approach, particularly older games, would be immensely useful: if this was viable on pre DX9/OpenGL2 hardware, it is likely to work fine for me.

One big question is in regards to this sort of effect:

(Image credit goes to Lume of lume.com)

Notice how the vertical fog gradation is continuous / smooth. OTOH, using textured quad layers, I can only assume that layers would be mighty obvious when walking through them -- the more sparse they were, the more obvious this would be. This is in contrast to where fog planes are aligned to face the player every frame, where this coarseness would be much less obvious.

Answer 1

Since you've asked for experiences, here are mine.

Back in the days when I was programming PS2 games, the "layered alpha quads" approach was a way that we often implemented fog. Occasionally as ground fog, but much more commonly as full-screen fog. And it worked just fine in either case. So yes, it was viable in the pre-fragment-shader days.

Well, sort of. The problem, as you've noted, was that if you want smooth fog such as that seen in the screenshot, you need a rather absurd number of alpha quads.

On the PS2, we could usually afford somewhere between three and five layers. Which definitely did look like "walls" of fog floating in front of you. More than that and the fill rate began to kill our frame rate.

Normally, these quads would be drawn at fixed distances in front of the camera, so you never got the situation that you mention, "walking through" one of them. On the other hand, by using those fixed distances, everything else in the world does pass through those planes as the player moves around, which is a pretty obvious graphical glitch. Almost everyone did it back then, but it wouldn't be acceptable now (unless you were doing it for stylistic reasons). (Exception: some folks calculated fog values as part of the PS2's equivalent of a vertex shader. That worked and was much faster, but required your models to be highly tesselated. You couldn't have long walls, for example, because fog was only being calculated at the corners of the wall, and then being smeared around over the whole face of the wall. The wall appeared fully fogged if you were standing right next to its middle, for example, since it was only testing the fog levels at its endpoints)

Note that if you place fog quads statically in the world (as you mention as a possibility), then you won't be able to get the appearance of extremely smooth fog, as in the image you provide -- there will be odd overlaps between adjacent quads depending on the viewer's orientation. Those overlaps may appear as stripes or trapezoids (if the quads are untextured) or as clumps (if they're textured).

But let's assume that we're using wide, screen-facing quads to do this ground fog and do some calculations for how to make really smooth fog using this method, on flat ground, with a camera looking straight forward -- that's our ideal situation. Let's assume HD resolution: 1920x1080, which puts our horizon at scanline 540. Let's also assume that we have visibility all the way to the horizon (that is, assuming that you don't have the fog reaching full opacity before reaching the horizon). With one fog quad starting and one stopping at each scanline (in order to get smooth fog), we need (540 * 2 ==) 1080 fog quads. Each of these 1080 fog quads will cover the whole horizontal expanse of the screen, and some vertical portion of the screen (the closest fog planes will touch just a couple rows of pixels at the bottom of the screen. The middle ones will cover more rows of pixels, the furthest ones will cover a smaller number of rows of pixels, due to perspective).

Let's estimate low, and say that on average, a fog plane will cover about 300 rows of pixels. The closest ones will cover less, the furthest ones will cover less, the middle rows will cover a lot more.

With that estimate, we get (1920x300 ==) 576,000 pixels being touched by the average fog quad. In total, that's (576,000 * 1080 ==) 622,080,000 pixels being touched in total for the whole "smooth fog via rendering lots of translucent geometry" effect. And that number will go up for people running at a higher resolution. And what's more, we get that same number of tests against the z-buffer, and nearly that same number of pixel blend operations, since all these transparent layers are drawing over each other again and again. That's a lot of pixels.

And that's the best case scenario -- you'll get far more screen coverage of the fog quads if the user looks down or crouches.

Note that since we're overlapping 1080 quads, we probably want an alpha value of about (1.0 / 1080 ~=) 0.0009 set on each, so that our fog reaches full opacity if you look through all 1080 quads. (We could go higher than that, but this is the value assuming that we want to spread the range around as much as possible). Note that this value can't be represented as the alpha component of a 32-bit color value (256*0.0009 ~= 0.237 and so will be rounded down to 0 if you try). You'll need to provide the 0.0009 value to OpenGL as a floating point value for this to work at all. (Also note that you won't actually want to set this same value on each -- while we defined that we wanted one quad to start and one to finish on each scanline below the horizon in order to give us smooth fog, you actually want the close quads to have lower opacity and the distant ones greater opacity, since the distance covered by a scanline is greater further away, and thus those distant fog quads represent more space than do the closer ones)

Also note that the fog blends won't work quite correctly using this approach, as they would with a modern shader -- instead of getting one calculation of "blend between the base object color and the fog color using this percentage", you get 1080 calculations of "blend between the color so far and the fog color by the fog percentage". Which means that the fog is going to affect objects following a logarithmic falloff. (That is, an object affected by 20 fog quads will appear less than twice as fogged as something affected by 10 fog quads, because the first fog quads have more of an impact in the blending operation).

All of which is to say: Please just use a fragment shader.

No, really. It's simpler and cheaper and faster and quicker to implement and less error-prone and lets you get back to actually making your game and is better in every possible way. We totally would have done it back in the PS2 era if it was even vaguely possible at the time.