I’m having a problem trying to fix my 1bit ordered dithering shader.

This shader is going to be perfect to be used but there is some detail that I would like to fix.

First of all, this shader is not just a shader, I'm going to make it for a videogame, so I don't need it to be smaller, with very few characters etc, but useful.
For this reason I used a lot of uniform variables, to let them be like a bridge between the shader and the game.
In this way some parameters could be dynamic and not just static, so that I can animate them or change them by pressing a button.

This below is my final shader for now. I studied a lot of variants by looking at ordered dithering in wikipedia and studying almost all the dithering shaders in shadertoy. None of them are useful because I'm making this game in game maker studio 2 and some ways to write a shader are not supported. So I had to understand very well how it behaves and try to make my own shader

// Ordered dithering aka Bayer matrix dithering
varying vec2 v_vTexcoord;
varying vec4 v_vColour;

uniform vec2 resolution;

uniform float pixelSize;
uniform float ditherSize;

uniform float gamma;
uniform float contrast;

uniform int invert;

uniform int vignette;

uniform float vsize;
uniform float vouter;
uniform float vfalloff;
uniform float valpha;

uniform sampler2D textureImage;

void main()
    vec2 uv = gl_FragCoord.xy / resolution.xy;
    vec2 tuv = gl_FragCoord.xy / 8.0 / ditherSize;
    vec2 pseudoPixel = floor( gl_FragCoord.xy / pixelSize );
    vec2 pseudoResolution = floor( resolution.xy / pixelSize );
    vec2 pseudoUv = pseudoPixel / pseudoResolution;
    vec4 color = v_vColour * texture2D( gm_BaseTexture, pseudoUv );
    float alpha = 1.0;
    tuv = fract(tuv);
    vec4 tdither = texture2D( textureImage, tuv );
    vec4 lum = vec4(0.299, 0.587, 0.114, 0);
    float center = vsize - distance(vec2(0.5),uv);
    if(vignette == 1) {
        vec3 vcol = vec3(smoothstep(vouter, vouter + vfalloff, center));
        color.rgb = mix(color.rgb, color.rgb * vcol, valpha);
    float dither = dot(tdither, lum);
    float grayscale = dot(color, lum) * gamma;
    grayscale = (grayscale - 0.5) * contrast + 0.5;
    vec3 col = vec3(step(dither,grayscale));
    if(invert == 0) {
        gl_FragColor = vec4(col, alpha);
    } else {
        gl_FragColor = vec4(1.0 - col, alpha);

As you can see here I'm using a texture for the bayer matrix, for different reasons

  • is more light to manage for the computer instead of computing a matrix
  • is more easy to do, because I don't have to know how to write matrices in glsl, in game maker studio 2 and I don't have to manipulate them to get the final texture.

In game maker studio I was not able to write double arrays like this

    //1Bit ordered dithering patterns
    vec3 dither_8(vec2 pos, float val) {
        const int mat[] = int[](
        0, 48, 12, 60, 3, 51, 15, 63,
        32, 16, 44, 28, 35, 19, 47, 31,
        8, 56, 4, 52, 11, 59, 7, 55,
        40, 24, 36, 20, 43, 27, 39, 23,
        2, 50, 14, 62, 1, 49, 13, 61,
        34, 18, 46, 30, 33, 17, 45, 29,
        10, 58, 6, 54, 9, 57, 5, 53,
        42, 26, 38, 22, 41, 25, 37, 21

or like this

    int dither[8][8] = {
            { 0, 32, 8, 40, 2, 34, 10, 42   },
            {48, 16, 56, 24, 50, 18, 58, 26 },
            {12, 44, 4, 36, 14, 46, 6, 38   },
            {60, 28, 52, 20, 62, 30, 54, 22 },
            { 3, 35, 11, 43, 1, 33, 9, 41   },
            {51, 19, 59, 27, 49, 17, 57, 25 },
            {15, 47, 7, 39, 13, 45, 5, 37   },
            {63, 31, 55, 23, 61, 29, 53, 21 } 

So, I decided to

  1. try sample a texture of a bayer matrix 8x8 px
  2. try to put on top of my source image (game canvas)
  3. multiply the texture to cover the whole canvas
  4. and render a step function to have only black or white

The shader works to not just have the dithering but to manage the single parameters to change the image dithered. In this way I can have different effects which make the game more dynamic.
Some of these are:

  • gamma to get the image darker or lighter
  • contrast to get the image more vivid
  • pixelSize to get some pixelization effects
  • invert to just invert the whole game colors

IMPORTANT! v_vColour is important to be multiplied by the final color, to keep let game maker still use his blending feature, game maker studio use 2D sprites but could be used for 3D also, so it manage the sprites like them are made by a triangular 3D plan in an orthographic view, so v_vColour is the vertex color of the plan where there are the sprites, in this way you can blend the sprites with a color by gml. Is very useful to have and I would not leave also if my final render is black and white, the reason is pretty simple, if I use white sprites I can always make them black multiplying black on them, but without using the shader for it, in this way I can use the shader for the whole game canvas, but i can use also some gml to fix here and there.

I got a better shader adding a vignette effect with lots of uniforms to change all parameters dynamically so:

  • in this way I can first have a vignette effect which change a little the edges of the game canvas giving more variants to all the sprites
  • I can enable/disable a lamp if I need one
  • I can mix the vignette effect and the gamma/contrast to have more image control
  • I can animate the lamp if I need it making the lights breath and having this way a more dynamic final effect

For the vignette I decided to make dynamic every parameter:

  • vignette on/off
  • vignette size
  • vignette outer edge
  • vignette falloff
  • vignette alpha

At a first moment I had the problem of the dithering size but I recently found how to fix it keeping the texture resolution but giving a third parameter to consider.

vec2 tuv = gl_FragCoord.xy / 8.0 / ditherSize;

ditherSize like every other parameter is dynamic to let me change the dither size in game. At this point the shader is going to be perfect, but in some points the shader is going to cut in half some "pixels" of the bigger bayer matrix texture, with the step function. The final effect would be with a perfect dithering with squares black or white, not cut half.

I should try to leave this cut

This is the artifact that I got


As you can see the texture is applied on top of the surface and rendered in a wrong way.

My dither size works perfectly on the 426x240 resolution canvas but changes when I get the full screen. I found a good dither size of 5.0 at full screen but it still cut some dither pixel

I hope to have been very good explaining this issue because is not an easy task. I hope this could be useful for you guys to understand more about dithering and in this particular case bayer matrix and ordered dithering and I really hope you can help me fixing it to make it perfect.

Thank you!

  • \$\begingroup\$ If I understand correctly - and pictures would really help understanding the issue - the problem is that you are scaling the texture with the pattern to fit fixed number of times on the screen, and that means that it does not always have an integer number of pixels. Well, if you need the number of pixel to be integer, round it (edit: to the next multiple of the pixel size, which you already have in an uniform, I think). So the pattern is cut off at the edge of the screen, but every texel is an integer number of pixels. \$\endgroup\$
    – Theraot
    Nov 29 '21 at 19:04
  • \$\begingroup\$ It sounds like you might be dealing with something similar to this previous Q&A - check to see if the answer there helps you at all. \$\endgroup\$
    – DMGregory
    Nov 29 '21 at 22:26

You must log in to answer this question.

Browse other questions tagged .