How to do color post processing in WebGL, when you can not read the color of the current pixel?

Question

In a WebGL 2 GLSL fragment shader, one can not access the pre-existing color value of the current pixel, i.e. the color that is already there in the framebuffer before the pixel that is currently calculated. I am woundering how one can get around this limitation?

For instance, let's say that I want to do some color post processing by running a fragment shader on a quad covering the whole screen. I'll use gamma correction as an example, but it could be any arbitrary function that takes a color for and return a new color. I would like to write code like this:

#version 300 es
precision highp float;
 
in vec2 uv;
out vec4 outColor;
 
void main() {
  vec4 currentColor = readPixel(uv)
  outColor = pow(currentColor, vec3(1.0, 1.0, 1.0) / 2.2);
}

However, there is no such function as readPixel, so I do not know what color I am trying to gamma correct. I can think of a few approaches to tackle the problem:

• Leverage the blend modes. However, they are quite limited (all are linear, I think), and I don't think they will help me here.
• Include the post processing in all fragment shaders. If all fragment shaders that put pixels on the screen has the same post processing built in at the end, I don't need to do it in a separate pass. But this complicates my code, since every single fragment shader is suddenly dependant upon my post processing needs. It's not so much post any more.
• Use ping pong buffers. Render to one framebuffer, and read the pixels from a texture belonging to another framebuffer. The non-existing readPixel can then be replaced with the very much existing texture function. However, this can't be great for performance since I am basically copying the full frame each frame.

What is the best path forward here? Is there a fourth option that I am missing? Is there some established best practice in a situation like this?

Answer 1

This is not a WebGL 2 restriction. This is fundamentally how GPUs and their 3D rendering APIs all work.

Because they're designed to render many triangles and fragments in parallel, they structurally eliminate data dependencies between these operations. A particular chunk of memory is either read-only (texture) or write-only (frame buffer / render target) for the full rendering pass.

If you want to change to read some memory, all pending writes to it need to finish first (a full pipeline flush). This would be unimaginably slow if it had to happen for every pixel — your super fast parallel processor would effectively become serial. So the APIs are designed so you can't write code that could incur such a serial penalty.

(There are some exceptions involving compute shaders, unordered access views, etc., but they're more specialized, and generally not a performant way to do a big bulk operation like a full-screen post effect where you know you want to write one value for every screen pixel)

Every post-processing system you've ever seen — yes, even that one that ran at ridiculously high frame rates — works using the "ping pong" approach you describe. You render the scene to one buffer, then read that buffer to apply post effects and write them to a second buffer, which is ultimately shown to the screen.

Your assumption that this would be unreasonably slow due to "copying" is not quite right. Whether writing to a new buffer or overwriting the value in-place, we still have to do the same work:

1. Sampling a buffer

2. Calculating a new colour based on the value read

3. Writing that colour to a buffer

The only difference is whether the write destination is the same as the read destination. These are cached separately on GPUs so you don't incur a performance penalty from writing to a different spot of memory — in fact you gain performance because there's no need to interleave loads and stores from the same memory chunk.

In fact, this can even give you more control over performance by allowing you to render your scene at a reduced resolution when needed, then apply a post-process resolve to map it to a full-res buffer to present to the screen, in a more sophisticated way than a bilinear upscale. "Dynamic resolution", checkerboard rendering, DLSS, and other strategies for getting high framerates at high resolutions work in roughly this way.

Modern games will often have many of these off-screen buffers, used for things like deferred rendering G-buffers, previous frames' output for motion blur, screenspace reflection, or temporal antialiasing, downsampling the frame buffer for fast bloom blurs, and all sorts of other effects.

It's not free, but generally fill rate (the number of times you write a value out — whether to the same place or a new one) is more likely to be a bottleneck on modern GPUs than the amount of memory used for render targets.