So I made a Mandelbrot generator in C# and I'm learning about HLSL and custom shaders in XNA (For a college course). I was hoping to do the mandelbrot calculations on the graphics card to make them faster. This is the code that I have so far (only part of my .fx file).

// sx, sy is the upper left corner of the mandelbrot coordinate area
// sw,sh is the width and height of the mandelbrot coordinate area.
float sx = -2.5, sy = -1, sw = 3.5, sh = 2;
float width, height; // Size of the quad I'm drawing this to

float4 PixelShaderFunction(VertexShaderOutput input) : COLOR0
    // TODO: add your pixel shader code here.
    float count = 500;
    // Convert from the drawing space to madelbrot space
    float tx = (input.UV.x / width) * sw + sx;
    float ty = (input.UV.y / height) * sh + sy;
    float tsx = tx, tsy = ty;

    // Find the divergence count
    for(float i = 0; i > 500; i++)
        // Get the next location
        tx = tx * tx - ty * ty + tsx;
        ty = 2 * tx * ty + tsy;
        // This is the part I'm having trouble with
        count = (tx * tx + ty * ty <= 4 ? count : min(count, i));
    // Calculate and return a color based on the count obtained
    float factor = count/500;
    return float4(1-factor, 1-factor, 1-factor, 1);

I think the issue has to do with having a ternary operation inside of a loop. The issue is that "factor" always results in having a value of 1, which means count obtains a value of 500. So my looping does nothing. I believe that my code is theoretically correct.

Maybe HLSL doesn't deal with the ternary operator very well? Does anyone have information about how they work or any limitations when using them?

Any comments are welcome.

  • 1
    \$\begingroup\$ if you are worried about the ternary operator then maybe replace it with an if else statement and check if that fixes things? \$\endgroup\$ May 5, 2014 at 23:49
  • \$\begingroup\$ You've written i > 500 instead of i < 500 in your loop. \$\endgroup\$ May 6, 2014 at 20:04

2 Answers 2


Your loop has a bad conditional. It is never true, so the loop is never entered. This:

for(float i = 0; i > 500; i++)

Should be

for(float i = 0; i < 500; i++)

(Change > to <.)

I haven't run your code, so I'm not sure if there are other issues.

I've compiled your code, and it seems to work with the above fix (I'm not sure if the maths is right, but it displays something nice).

If you're using ps_2_0 you have to crank the number of iterations down ridiculously low (6 instead of 500) to fit in the instruction limit.

If you switch to ps_3_0 (the highest shader model XNA supports), you can do a few more iterations thanks to the higher instruction limit. But shader model 3 also supports real looping, which allows for significantly more iterations (~250). Decorate (MSDN) your loop like so:

[loop] for(float i = 0; i < 250; i++)

To use shader model 3, you'll need to enable HiDef mode. If you do what I did and draw with SpriteBatch, you'll need to follow these instructions to supply a vertex shader.

  • \$\begingroup\$ Another way to eke out more iterations is to use a floating point rendertarget to save an intermediate position and iteration count at the end of your loop. You can blit it through the shader many times, and then run the final colorization step on it when you've done as many iterations as you want. \$\endgroup\$
    – DMGregory
    May 6, 2014 at 12:33
  • \$\begingroup\$ Wow. Derp. Thank you. I was trying to go in reverse (from 500 to 0) so I had a > with a bound of 0, so when I reverted the change I must have just changed the bound and not the >. Sorry about that. I'll look into the other details you explained and see if I can get it to work the way I want. Thanks! \$\endgroup\$ May 6, 2014 at 16:44
#define Iterations 254
float2 Pan;
float Zoom;
float Aspect;    
float4x4 MatrixTransform;

void SpriteVertexShader(inout float4 color    : COLOR0,
                    inout float2 texCoord : TEXCOORD0,
                    inout float4 position : SV_Position)
{    position = mul(position, MatrixTransform);    }

float4 Pixel_Shader(float2 texCoord : TEXCOORD0) : COLOR0
    float2 c = (texCoord - 0.5) * Zoom * float2(1, Aspect) - Pan;
    float2 v = 0;

[loop] for (int n = 0; n < Iterations; n++)
    { v = float2(v.x * v.x - v.y * v.y, v.x * v.y * 2) + c;   }

    return (dot(v, v) > 1) ? 1 : 0;

    VertexShader = compile vs_3_0 SpriteVertexShader();
    PixelShader = compile ps_3_0 Pixel_Shader();


Does a very decent monochrome pass using the 255 loops. Props to Shawn Hargreaves, XNA4.0 (PS_3_0) ready version mangled by me.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .