Taking your example, you have a step function of the distance, which produces a perfectly hard (aliased) edge. A simple way to antialias the circle would be to turn that into a soft threshold, like:
float distFromEdge = 1.0 - dist; // positive when inside the circle
float thresholdWidth = 0.01; // a constant you'd tune to get the right level of softness
float antialiasedCircle = saturate((distFromEdge / thresholdWidth) + 0.5);
return lerp(outsideColor, insideColor, antialiasedCircle);
Here I used a clamped linear ramp for a soft threshold function, but you could also use
smoothstep or something else. The
+ 0.5 is to center the ramp on the mathematical location of the edge. Anyway, the point is that this function smoothly changes from
insideColor over some range of distances, so if you pick
thresholdWidth appropriately you'll get an antialiased-looking edge.
But how should you choose
thresholdWidth? If it's too small, you'll get aliasing again, and if it's too large the edge will be overly blurry. Moreover, it'll generally depend on camera position: if
dist is measured in world-space or texture-space units, then a
thresholdWidth that works for one camera position will be wrong for another.
Here's where the screen-space derivatives come in (yes, they're the
ddy functions as you guessed). By calculating the length of the gradient of
dist you can get an idea how rapidly it's changing in screen space and use that to estimate the
float derivX = ddx(distFromEdge);
float derivY = ddy(distFromEdge);
float gradientLength = length(float2(derivX, derivY));
float thresholdWidth = 2.0 * gradientLength; // the 2.0 is a constant you can tune
You still have a value you can tune to get the desired level of softness, but now you should get consistent results regardless of camera position.