# Depth of Field Blur - Weighted sampling?

I've been studying Intel's fantastic article titled "An investigation of fast real-time GPU-based image blur algorithms" (here), where-in they state that Gaussian blur would need to be customized for proper use in achieving a depth-of-field effect. The relevant part states:

They are all generic algorithms that are fully applicable to effects like a Bloom HDR filter. However, in a specific scenario, for example for use in Depth of Field, they would require additional customization (such as weighted sampling to avoid haloing/bleeding) which can impact relative performance.

However they don't go into detail about how to compute the weighted sampling? Does anyone know what this means, or know of relevant documentation / example code?

I have been using guassian blur for DoF and have indeed seen the bleeding and other artifacts, so I've been looking for ways to solve these problems.

These are the key things that would make Gaussian Blur very fast on the GPU:

• Separated horizontal and vertical passes
• Remove tail parts of a large Gaussian number since they contribute next to nothing to the target pixel
• Use binomial coefficients as Gaussian number since binomial coefficients (or Pascal's triangle) is the discrete form of Gaussian function
• Use bilinear sampling from hardware to reduce number of samples you sample from the texture

The first two are quite obvious and so I won't explain them in detail. A good suggestion on how to "chop off" the tails of Gaussian numbers would be anything smaller than 1/255.0 because each channel is likely from 0 - 255. The last two are some what less intuitive. Basically the reason you want to use binomial coefficients is because mathematically, as you get closer to the bottom of the Pascal's triangle, the closer you are to the Gaussian function. In fact, binomial distribution is normal distribution with p = 0.5. To convert back and forth, use:

sigma^2 = 0.25n
mu = 0.5n


where n is the number of terms in the binomial distribution. This tells us, if we want a Gaussian filter with say sigma of 10, we will need 400 term binomial distribution! The center number is actually so large (1.029525991*10^118) that the edge (1/(2.5822498*10^120)) would basically be zero. However, if we apply our threshold of 1/255.0, this suddenly drops from 400 terms to merely 43 terms! So in conclusion, you want to use binomial distribution because:

1. The human eye cannot tell the difference between sigma of 1 and 1.75 when blurring with Gaussian. This means we can skip them. This works well in using binomial distribution because we don't need it to be continuous.
2. You can adjust the number of terms (hopefully with a script) without visually tell a difference between a full-blown Gaussian filter V.S. reduced binomial filter.
3. Statically bake these terms into the shader (or code) and therefore no dynamic calculation is needed.

Finally the last optimization is actually a bit harder to explain ... We know that bilinear sampling simply means that the hardware fetch the pixel weighted by their texture coordinates. So if we ask for a color at texture coordinate that's between two pixels, we get 50% of each. The goal is to calculate a texture coordinate such that we get X% of left and Y% of right where X and Y are Gaussian (or binomial) weights.

To actually answer your question, to compute the weighted sample, you calculate the weights (like described above), and then you offset by that number of terms. Say you use 43 terms, that means you offset by 21 pixels left and 21 pixels right of the target pixel. Since we work in texture coordinates, we first calculate how wide each pixel is: (in GLSL)

//horizontal pass
vec2 texCoordOffset = vec2(1.0, 0.0) / vec2(TEXTURE_WIDTH, TEXTURE_HEIGHT);
//If vertical, the above should be vec2(0.0, 1.0)
vec4 outputColor = vec4(0.0, 0.0, 0.0, 0.0);
for(int i = 0; i < 21; ++i)
{
outputColor += binomWeights[i] * texture(TEXTURE_0, uv - vec2(1.0, 0.0) * binomOffsets[i]);
outputColor += binomWeights[i] * texture(TEXTURE_0, uv + vec2(1.0, 0.0) * binomOffsets[i]);
}


The weights are simply 1/sum_of_binomial; For example, a 5 term binomial distribution would be 1, 4, 6, 4, 1 and their sum is 16 so the weights should be: 1/16, 4/16, 6/16, 4/16, 1/16; The texture offsets should be: (2*(1/16) + (1*(4/16)) / (1/16 + 4/16). The 2 represents 2 pixels from target pixel and 1 represents 1 pixels from target pixel. This gives you a number in texture coordinate and in this case it is 1.2; That is if we fetch at 1.2 pixels away from the target pixel (center), we get 80% of pixel that's 1 away from center and only 20% of pixel that's 2 away from center.

I hope this isn't too confusing ...

• This provides a great overview of considerations in making a high-performance Gaussian blur effect. But it does not answer the question that was asked, which is how to compute sample weights in order to avoid/minimize bleeding and haloing artifacts when using the blur in a depth of field effect. To address this question, we need to take into account the relative depths of the samples, so that crisp in-focus objects' colours don't get picked up by blurry objects behind them, for example. – DMGregory Jul 28 '17 at 19:44
• Right, sorry about that. A simple implementation I thought of would be blurring the scene in two buffers. One is used for foreground and one for background. You can simply linearly interpolate between them. Depth 0 = 100% foreground blurred and say at depth of 0.25, you linearly transition to non-blurry scene. Then from say 0.65 to 1.0, you transition to 100% blurred background. Since you are learping, bleeding will be harder to see. – ChaoSXDemon Jul 28 '17 at 21:18
• Be careful here - a hasty solution can pose as many problems as no solution at all. To correctly handle depth of field blurring, you need to consider the circle of confusion at each sample point, and where your rendered pixel falls within that range. Because this changes continuously with depth, splitting the scene into a foreground and a background buffer is not sufficient to handle all of the depth variation within each of those slices. – DMGregory Jul 28 '17 at 21:37
• @DMGregory, that's a much involved technique and would naturally be more expensive. You are, of course, right that the simple method I said wouldn't account for all the depth variations. To correctly calculate CoC will pose a bit of a challenge and may not play well (easily) with other things, say SSAO. – ChaoSXDemon Jul 28 '17 at 22:27
• That's exactly why the Intel quotation in the question called it out as a significant source of additional complexity and cost. And that's why this question exists in the first place, because a method for computing appropriate weights for good depth of field blurring is not immediately obvious. It's not infeasible though — realtime Bokeh DoF blurs exist. They're typically applied after SSAO and similar post effects, just like the interpolated scheme you sketched above would be. – DMGregory Jul 28 '17 at 22:35