How do I counteract FXAA blur?

Question

FXAA is a cheap post processing anti-aliasing method which works great with deferred pipelines where traditional MSAA may not be a viable option. It's also easier to implement than alternatives such as SMAA.

Unfortunately, as a quick Google search will show, a lot of PC gamers hate it because even when setting the shader to its best quality settings it still adds a slight blur to the outputted image. This is particularly noticeable when a scene has textures with fine detail and/or small specular highlights.

Is there a way to counteract this blur without reintroducing significant aliasing?

Answer 1

Coming from a graphic design background, one solution I usually use for making blurry images appear sharper is to overlay noise. Of course, adding random noise does not look good. The noise must be relevant to the underlying material. The classic case is making a material look weathered in Photoshop by using grunge textures and setting their layers to multiply/dodge/burn/etc.

So, how do we take the image outputted by the FXAA shader and extract some relevant noise? Again, Photoshop has the answer. Some of oldest effects in the program such as blur, sharpen, edge extract and emboss are created with what are known as convolution matrices.

The matrix we're interested in has various names, though I've always known it as 'edge extract' and it looks like this:

 0, 0, 0
-1, 1, 0
 0, 0, 0

To apply this effect we want to take the output of the FXAA shader as input and render a fullscreen quad. Then in the fragment shader we sample 9 texels with the fragment position at the centre. Multiply each sampled colour by the corresponding number in the matrix, then add all of these multiplied samples together. Below is a sample fragment shader written in GLSL that achieves this:

#version 330 core
out vec4 out_colour;
uniform sampler2D blit_map;
uniform vec2 screen_size;

void main()
{
    vec4 sum = vec4(0.0);
    vec2 offset[9] = vec2[](vec2(-1.0, 1.0), vec2(0.0, 1.0), vec2(1.0, 1.0), 
                            vec2(-1.0, 0.0), vec2(0.0, 0.0), vec2(1.0, 0.0), 
                            vec2(-1.0, -1.0), vec2(0.0, -1.0), vec2(1.0, -1.0));
    float kernel[9] = float[](0.0, 0.0, 0.0, 
                             -1.0, 1.0, 0.0, 
                              0.0, 0.0, 0.0);

    for (int i = 0; i < 9; i++)
    {
        vec4 colour = texture2D(blit_map, (gl_FragCoord.xy + offset[i]) / screen_size);
        sum += colour * kernel[i];
    }

    out_colour = sum;
}

If all goes correctly you should end up with something that looks like this (the original image was a height-mapped grassy terrain with some trees):

Here's the same picture with exaggerated lighting (so we can see what's going on):

We now have some noise relevant to our underlying image, so lets add it back onto the original output we received from the FXAA shader.

#version 330 core
out vec4 out_colour;
uniform sampler2D blit_map;
uniform vec2 screen_size;

void main()
{
    vec4 sum = vec4(0.0);
    vec2 offset[9] = vec2[](vec2(-1.0, 1.0), vec2(0.0, 1.0), vec2(1.0, 1.0), 
                            vec2(-1.0, 0.0), vec2(0.0, 0.0), vec2(1.0, 0.0), 
                            vec2(-1.0, -1.0), vec2(0.0, -1.0), vec2(1.0, -1.0));
    float kernel[9] = float[](0.0, 0.0, 0.0, 
                             -1.0, 1.0, 0.0, 
                              0.0, 0.0, 0.0);

    for (int i = 0; i < 9; i++)
    {
        vec4 colour = texture2D(blit_map, (gl_FragCoord.xy + offset[i]) / screen_size);
        sum += colour * kernel[i];
    }

    float sharpen_amount = 0.25;
    out_colour = (sum * sharpen_amount) + texture2D(blit_map, gl_FragCoord.xy / screen_size);
}

You'll note we also scaled the sum before adding it onto the original colour. The blur caused by FXAA is subtle, therefore the sharpening should also be subtle so you'll want to keep the sharpening amount low.

You've probably noticed by now that this shader isn't as cheap as it could be. There are nine texture lookups, most of which add nothing to the final image as they're multiplied by zero. As a final step, let's optimize the shader.

#version 330 core
out vec4 out_colour;
uniform sampler2D blit_map;
uniform vec2 screen_size;

void main()
{
    vec4 colour = texture2D(blit_map, gl_FragCoord.xy / screen_size);
    vec4 sum = colour + (texture2D(blit_map, (gl_FragCoord.xy + vec2(-1.0, 0.0)) / screen_size) * -1.0);

    out_colour = (sum * 0.25) + colour;
}