This question already has an answer here:

(Related, but somewhat different, to my previous question)

How can I determine in a fragment shader if a fragment is in the shadow of a sphere?

That is, if it is occluded by the sphere and is past the sphere's horizon from the camera (if you are in front of the horizon you are not in the shadow even if you are in the sphere; the sphere is not solid)

In perspective, the horizon of the sphere is in front of the centre-point. Imagine holding a football at arms-length and stare at a point on the horizon of it; now move the football closer to you eye; what happens? It is no longer visible; the closer the sphere is to the eye, the less of the surface you can see:

enter image description here

As I imagine it, it is:

  1. are you in the cone that is from the camera and passes through the horizon of the sphere as seen from the eye? and

  2. are you past that horizon?

How do you compute the plane of the horizon, the cone, and how do you test for it in the fragment shader?

  1. Imagine you had a ray that was through camera and fragment. The nearest distance between that ray and the centre of the sphere being less than the sphere's radius would tell you if it was in the 'cone' of the sphere.

  2. Now imagine you knew the distance the camera to the horizon; if the closest point on the ray was less than this distance, its in front of the sphere; else its past the horizon. (We can make this simplification the fragments we want to test are never deep in the middle of the sphere.)

With these two values, you determine if a fragment is 'in the shadow' of the sphere.

But how do you compute this? What, even, is the coordinate of the camera (0,0,-1 if orthogonal projection, else 0,0,0?)? And how far away is the horizon of the sphere?

And what's the code for nearest point on ray to point? What I've come up with is [src]:

t = (P-B).(A-B) / (A-B).(A-B)

If P is the sphere's centre, and A is the fragment's position and B is the camera (at 0,0,0 so can be omitted as its a no-op):

// its a unit sphere:
var sphereCentre = mat4_vec3_multiply(

Then the vertex shader just has to pass the fragment position along:

precision mediump float;
attribute vec3 vertex;
uniform mat4 pMatrix, mvMatrix;
varying vec3 p;
void main() {
    gl_Position = pMatrix * mvMatrix * vec4(vertex,1.0);
    p = gl_Position.xyz/gl_Position.w;

And the fragment shader sees if its inside-the-cone using distance to sphere centre:

precision mediump float;
uniform vec4 fgColour, bgColour;
uniform vec3 sphereCentre;
uniform float sphereRadius; // always 1 in my game fwiw
varying vec3 p;
void main() {
    float t = dot(sphereCentre,p) / dot(p,p); // where on line?
    vec3 d = (p*t) - sphereCentre; // distance from nearest point to sphere
    //### now we need to know if its in front of the horizon to force fgColour ???
    gl_FragColor = (dot(d,d) <= sphereRadius*sphereRadius)? fgColour: bgColour;

This might be along the right track, but its not working (it looks hopeful drawn in ortho; in perspective it often draws things in the wrong colour). And how to compute the horizon?


marked as duplicate by Nicol Bolas, Sean Middleditch, Trevor Powell, Josh, bummzack Feb 23 '13 at 10:46

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • \$\begingroup\$ If your question is about geometry, why are you talking about shaders? \$\endgroup\$ – Ivan Kuckir Feb 21 '13 at 18:18
  • \$\begingroup\$ How is this any different from your other question? \$\endgroup\$ – Nicol Bolas Feb 21 '13 at 20:18
  • \$\begingroup\$ @NicolBolas objects that are not on the sphere surface but behind the sphere \$\endgroup\$ – Will Feb 21 '13 at 20:19
  • \$\begingroup\$ @Will: Oh. You should probably make it clearer in your question that you're not talking about fragments on the sphere. \$\endgroup\$ – Nicol Bolas Feb 21 '13 at 20:27
  • 1
    \$\begingroup\$ @Will So why doesn't your question just ask that -- how to draw something a different colour if it's being viewed through a non-rendered sphere? Why all the detail about "sphere horizons" and shadows and diagrams and stuff? It's like you decided on the answer you wanted before you asked the question, and are refusing all the simple and easy approaches people have offered, in favour of your pre-chosen complicated one. To say nothing of how confusing it gets when you simultaneously talk about the object being "occluded" by the sphere, but also that the sphere isn't actually being drawn. \$\endgroup\$ – Trevor Powell Feb 22 '13 at 7:25

The generalized solution would be to (in world space) draw a line between the fragment's location and the camera. If said line passes through or intersects the sphere your fragment falls behind the sphere and will not be visible.

Here is a breakdown of the math you would use for checking the intersection.

  • \$\begingroup\$ but the horizon of the sphere ... its not about culling; its about knowing what to draw as would be seen through the sphere. \$\endgroup\$ – Will Feb 21 '13 at 20:20
  • \$\begingroup\$ @Will: "but the horizon of the sphere" What about it? It's irrelevant. If you want to know if there is a sphere between the camera and a point in 3D space, you use ray tracing. If that's not the problem you're trying to solve, then you need to better explain that problem. Perhaps with a diagram that you didn't draw in MS Paint, something with labels that makes it clear where a point should be considered "visible" and where it should not. \$\endgroup\$ – Nicol Bolas Feb 21 '13 at 21:41
  • \$\begingroup\$ @NicolBolas "Imagine holding a football at arms-length and stare at a point on the horizon of it; now move the football closer to you eye; what happens? It is no longer visible; the closer the sphere is to the eye, the less of the surface you can see:" \$\endgroup\$ – Will Feb 21 '13 at 21:49
  • \$\begingroup\$ @Will: ... and? In what way does ray tracing not capture this? Pick a point on a sphere. See what the ray that goes to that point goes. Does it go through the sphere? If not, move the sphere closer. See how it eventually goes through the sphere? \$\endgroup\$ – Nicol Bolas Feb 21 '13 at 21:50
  • 2
    \$\begingroup\$ @Will: You don't need to compute the horizon! It happens implicitly by simply raytracing in your fragment shader. If a fragment can "see" the camera from it's position, then it's not shadowed by the sphere. If a fragment cannot, then it is shadowed. Period. You never need the exact plane or anything other than a simple ray-sphere test. And even then, you don't need most of the test; just check the determinant to see if it's negative. If it is, then the fragment is visible. \$\endgroup\$ – Nicol Bolas Feb 22 '13 at 0:01

It suddenly occurs to me that there's a really easy solution to all this, which requires no shader logic at all. Just use the depth buffer.

Draw a (filled in) circle at the proper location and distance from the camera, pointed straight at the camera. Just use the glColorMask to turn off depth writes when you do draw the circle.

Once the depth is laid down, any fragments you draw later that are behind it will be culled by the depth buffer.

I'll assume you know how to draw a filled-in circle of a given radius, location, and facing. So instead, I'll cover how to compute where it goes.

Given the world-space position of your sphere P, with a world-space sphere radius R, and a camera location in world-space C, the world-space location of your circle should be this.

We compute the distance from P to place our circle. This is done with this equation:

dist = (R*R) / length(P - C)

We then use the distance to compute the the actual location, by getting the direction from P towards C and multiplying it by the distance.

dir = normalize(C - P)
pos = dir * dist

Note that dir is the direction towards the camera which the circle plane must face.

Oh, and don't forget to turn the color mask back off after you draw the sphere.

  • \$\begingroup\$ The sphere is not solid; I do not want to cull the fragments behind it, I want only to change their colour. Drawing the whole scene twice - once in background colour, then writing the z of the horizon, then drawing it all again in the foreground colour would possibly work but could lead to blending artifacts from the overdraw? \$\endgroup\$ – Will Feb 22 '13 at 6:53
  • \$\begingroup\$ @Will: You can't have blending artifacts if there's no blending. As long as everything is done invariantly (ie: you use the same vertex shader code), you should be fine. \$\endgroup\$ – Nicol Bolas Feb 22 '13 at 7:21
  • \$\begingroup\$ If I draw something in the 'back colour' and then again in the 'front colour', it will be a different colour than if I had not drawn the back colour under it. \$\endgroup\$ – Will Feb 22 '13 at 7:42
  • \$\begingroup\$ @Will: If you write a color to the framebuffer, you write the color to the framebuffer. If you're seeing different behavior, then you have blending on or aren't writing what you think you are. \$\endgroup\$ – Nicol Bolas Feb 22 '13 at 8:37
  • \$\begingroup\$ theres an alpha-channel. And the GPU does anti-aliasing etc. \$\endgroup\$ – Will Feb 22 '13 at 8:50

To answer my own question:

The camera is at 0,0,0 in view space.

The sphere's centre has to be converted to view space and passed in as a uniform; this means multiplying it by the modelview matrix.

The vertex shader passes on the fragment's coordinate in view-space:

precision mediump float;
attribute vec3 vertex;
uniform mat4 pMatrix, mvMatrix;
varying vec4 pos;
void main() {
    pos = (mvMatrix * vec4(vertex,1.0));
    gl_Position = pMatrix * pos;

The fragment shader does the check:

precision mediump float;
uniform vec4 fgColour, bgColour;
uniform vec3 sphereCentre;
uniform float sphereRadius2; // always 1 in my game fwiw
varying vec4 pos;
void main() {
    vec3 p = pos.xyz/pos.w;
    float t = dot(sphereCentre,p)/dot(p,p);
    vec3 d = (p*t)-sphereCentre;
    gl_FragColor = (dot(d,d) > sphereRadius2 || t<=1.0)? fgColour: bgColour;

This computes where on the fragment's ray the nearest point is, as a ratio t.

It also computes the distance (squared - dot(d,d)) between the nearest point and the sphere's centre.

If the nearest point is beyond the sphere or the intersection of the line is less than 1 (meaning it is in front of the sphere) then it is in the foreground.

To compute point sprites correctly you need an extra check if it is in the background because the check outlined above is for the centre-point of the sprite and not each fragment in the sprite. If its possibly in the shadow of the sphere, you need to additionally check the exact fragment in the sprite.


Not the answer you're looking for? Browse other questions tagged or ask your own question.