I'm trying to implement SSAO together with deferred shading. The deferred shading works just fine, and so does calculating the AO as seen in the first image below. However, taking AO into consideration when combining the light with gbuffer data produces color banding as seen in the second figure.
The third image shows a result after messing around, ending up multiplying the final color by 3, increasing the overall light intensity. The light banding is somewhat alleviated, and further increasing the light intensity reduces banding even more. This is of course not a solution but perhaps a step in the right direction.
Nevertheless it got me thinking. For the resulting texture I'm using an RGBA8_UNORM format, which gives me 256 distinct values for the various channels, right? I store the AO in a single 8 bit channel texture. The only light affecting the mesh is ambient light, and the color of the mesh is a single shade of green. Doesn't this mean that the already low bits of the render target have to store even more shades when AO is applied? This makes sense considering that increasing the precision of the AO map doesn't fix the problem, but not so much when simply increasing the intensity of the final color reduces the banding. How is this generally handled when using deferred shading? I should note that attenuating the final color with the normalized horizontal pixel value in [0,1] produces similar banding, which is perfectly vertical instead.
Can it be something that has to do with gamma correction (I don't explicitly do it myself)? Is that something one must do when using deferred shading?
It turned out that DXUT made the back buffer SRGB (meaning that it handles gamma correction), but gamma was never corrected at the beginning, resulting in incorrect gamma handling. Correcting gamma while sampling textures resulted in clearer, but darker colors. Increasing ambient light to compensate for the darkness resulted in less banding while still having dimmer surroundings. Another bonus is that the colors don't look as washed-out as before.
It's not completely perfect however, but maybe HDR rendering can take care of that (big maybe, I don't much about it at all). Also, colors of the mesh are constant, which probably contributes. I believe having high-frequency textures makes the little banding that's left less apparent.