Unstable sampled color in GLSL shader

Question

It looks like that sampling the same exact color data from different areas of a texture atlas results in different vec4 in the GLSL fragment shader.

In other words, texture2D is directly dependent of the specified texture coordinates, and not only transitively because of the color of the texel at those coordinates.

To say that with an example, I am getting the impression that sampling a red pixel surrounded by blue pixels from, say, (0.1, 0.2) may give a different result than sampling the same shade of red surrounded by the same shade of blue at, say, (0.7, 0.3).

Is this non-deterministic behavior that I am observing likely to be real?

Additional details

I have a atlas that we use to texture multiple screen-aligned quads. Each vertex has a xy position and uv texture coordinates. The allotted rectangles in the atlas do not overlap; furthermore, a 2 pixel margin exists between adjacent rectangles. We are using linear filtering and no mipmapping.

The atlas is populated dynamically using sprites downloaded from the web. Two different runs of the program may have the atlas populated in different orders, even if user input is exactly the same, due to the fact that some HTTP requests may complete before others.

So now imagine that on a run we have a Princess Peach sprite in the middle of the screen; the texture is coming from the upper left corner of the atlas, because Princess Peach was the first sprite added to the scene.

In another run, there have been a few sprites loaded but now their quads are not on the screen and again we have Princess Peach in the middle, in the same exact position as the first run. But this time the texture is coming from a different region of the atlas.

Problem: rendered images are not the same. The differences are imperceptible and usually consist of only a few pixels near the edges of the rendered sprites. The differences are detected by a screenshot testing framework. Where is the source of randomness?

Even more details

• It's a WebGL project.
• It happens on both GeForce and Quadro cards.
• Using nearest neighbor sampling leads to more stable (and ugly) renderings but it does not fully eliminate the problem.
• Enforcing a strict order when copying the sprites to the atlas fully solves the problem, but it is not an acceptable solution.
• I inspected the GL state using an inspector and I found two frames that should have been the same in two different runs of the program; the state is the same in both cases, the only difference being a different arrangement of sprites in the atlas, and consequently different texture coordinates in the array buffer.
• I know that I could test using some sort of fuzzy/thresholded comparison but first I would like to rule out bugs on my side.

Answer 1

Two possibilities I can think of so far:

1. Contamination in transparent regions. You mention the artifacts show up mainly near the edges of the sprites. Depending on how you perform your atlas updates, it could be that when the sprites are placed in the atlas, only the texels with non-zero alpha are being replaced. Texels with zero alpha could, under some update methods, retain the RGB value of whatever was in that part of the atlas previously (eg. a previous sprite, or uninitialized video memory...). This can cause odd fringing at the edges of objects under bilinear texture filtering, as the RGB is tugged away from your sprite's value while the alpha is still ramping down.

2. Rounding/phase differences. Especially if your atlas packing uses a non-power-of-two grid of entries, or you're scaling your sprites as you copy them into the atlas. Factors like these introduce a multiplier into the texture coordinates used to read or write into the atlas, which could lead to a sub-pixel-sized offset that varies from entry to entry in your atlas with a period different than your entry spacing. So moving the same sprite over one slot causes the offset to hit it in a different phase, slightly nudging texel lookups away from where they landed in the original position.

I think we can rule out floating point differences. Most GPUs perform texture filtering using fixed-point arithmetic with 8 binary digits of sub-pixel precision, and 32-bit floating point texture coordinates have even more. Unless you're using a reduced-precision format for the UV coordinates, this shouldn't come into play.

To narrow this down further, it would help to see the methods you use to initialize & populate the atlas, and to produce your textured quads. Image examples of the effect would help too - particularly if you can construct clear-cut cases using eg. grid textures or other regular patterns that will help us isolate which pixels/cases contribute to the issue.

Answer 2

Texture coords (like pixel coords) go from the edges of pixels. So, imagine you have a 3x1 texture.

0.0                           1.0
 |                             |
 V                             V
 +---------+---------+---------+
 |         |         |         |
 |         |         |         |
 |         |         |         |
 +---------+---------+---------+

So if you're trying to select a specific pixel the correct texture coordinate for that pixel is

 uv = (pixelCoord + 0.5) / size;

Plugging that into the example above the correct texcoords for each of the pixels are

 pixel 0 =  (0 + .5) / 3 = 0.16666
 pixel 1 =  (1 + .5) / 3 = 0.5
 pixel 2 =  (2 + .5) / 3 = 0.83333

That may or may not have anything to do with the issue you're seeing. I just know sometimes people think it's

 uv = pixelCoord / size

That ends up putting you exactly on the boundary between 2 pixels and so if your filtering is set to NEAREST you'll often get randomly one pixel or the other depending on various rounding issues and if you are using LINEAR you'll get the exact mix between 2 pixels.

For mapping textures to pixels it works because if your source is 3x1 and your dest is 3x1 then the grids overlap perfectly and GL will do what you expect. But as soon as you start trying to pick specific pixels out of the texture then it starts to get more complicated.

Being that it's WebGL it seems like it would be easy to post a simple example. Make a texture using canvas2d. Put a red block and and a blue block in the texture. Make another, put the red and blue blocks in different places that repo your issue. Render with those as your atlas. That would help narrow the issue down.

Answer 3

Our current understanding of the situation is:

The sampler will do whatever the heck he wants

The smallest code that can reproduce a similar looking issue is a quad rendering app that does texture mapping this way:

varying highp vec2 v_texcoord;

uniform highp sampler2D u_texture;
uniform highp float u_time;

void main(void) {
    highp vec2 err = 10.0 * vec2(cos(u_time), sin(u_time));
    highp vec2 iErr = floor(err);
    mediump vec4 color = texture2D(u_texture, v_texcoord + iErr);
    gl_FragColor = color;
}

The brick texture used in this test is taken from TextureX.com website.

If you use GL_REPEAT texture wrapping, this fragment shader should produce the same output every frame. There are no possible rounding errors issues/excuses. Every frame surely looks the same to the naked eye. But if you take a screenshot and enhance the difference in the RGB channels, you will see that there are a lot of mismatches. Probably imgur.com is doing a recopression of the image and they end up showing as vertical streaks of white blobs, but nevertheless, they are not part of the original texture.

There is also a https://www.shadertoy.com/view/XdfBzM that shows the problem in motion.

Note for people that need to do screenshot testing

If your app dynamically populates a texture atlas, and the order to which sprites/glyphs/textures/whatever are added to the atlas is somewhat pseudo-random, you may want to provide also a deterministic path to use during testing, or simply accept the fact that results are going to be different every time and find a threshold value that works for you.