# How do I correctly project a point behind the camera?

I am making a 3D game in which I put an exclamation marker above points of interest.

In order to find out where in the 2D screen should I put my marker, I am manually projecting the 3D point where the marker should be.

It looks like this:

Looks pretty good. When the marker is outside of the screen, I simply clip the coordinates so they fit into the screen. It looks like this:

So far the idea is going pretty well. However, when the points of interest are behind the camera, the resulting X,Y coordinates are inverted (as in positive/negative), and I get the marker to appear on the opposite corner of the screen, like this:

(The projected-then-clamped point is the tip of the marker. Don't mind the rotation of the marker)

It makes sense, since the coordinates behind the frustum are inverted in X and Y. So what I'm doing is to invert the coordinates when they are behind the camera. However, I still don't know what is exactly the condition when the coordinates should be inverted.

This is currently what my projection code looks like (in C# with SharpDX):

    public override PointF ProjectPosition(float viewportWidth, float viewportHeight, float y)
{
var projectionMatrix = Matrix.PerspectiveFovRH(GetCalibratedFOV(Camera.FOV, viewportWidth, viewportHeight), viewportWidth / viewportHeight, Camera.Near, Camera.Far);
var viewMatrix = Matrix.LookAtRH(new Vector3(Camera.PositionX, Camera.PositionY, Camera.PositionZ), new Vector3(Camera.LookAtX, Camera.LookAtY, Camera.LookAtZ), Vector3.UnitY);
var worldMatrix = Matrix.RotationY(Rotation) * Matrix.Scaling(Scaling) * Matrix.Translation(PositionX, PositionY, PositionZ);
var worldViewProjectionMatrix = worldMatrix * viewMatrix * projectionMatrix;

Vector4 targetVector = new Vector4(0, y, 0, 1);
Vector4 projectedVector = Vector4.Transform(targetVector, worldViewProjectionMatrix);

float screenX = (((projectedVector.X / projectedVector.W) + 1.0f) / 2.0f) * viewportWidth;
float screenY = ((1.0f - (projectedVector.Y / projectedVector.W)) / 2.0f) * viewportHeight;
float screenZ = projectedVector.Z / projectedVector.W;

// Invert X and Y when behind the camera
if (projectedVector.Z < 0 ||
projectedVector.W < 0)
{
screenX = -screenX;
screenY = -screenY;
}

return new PointF(screenX, screenY);
}


As you can see, my current idea is to invert the coordinates when either the Z or W coordinates are negative. It works -most- of the time, but there are still some very specific camera locations where it doesn't work. In particular this point shows one coordinate working and the other one not (the correct location should be the bottom right):

I have tried inverting when:

• Z is negative (this is what makes most sense to me)
• W is negative (I don't understand the meaning of a negative W value)
• Either Z or W is negative (what's currently working most of the time)
• Z and W are of different sign, a.k.a.: Z / W < 0 (makes sense to me. doesn't work though)

But have still not found a consistent way with which all my points are correctly projected.

Any ideas?

How about doing it a bit differently?

Start as usual and render all markers within viewport. If a marker is outside of the viewport - proceed:

1. Get positions of the marker A
2. Get position of the camera B (maybe slightly in front of it)
3. Calculate 3D vector AB between these two
4. Convert and normalize that vector into 2D screen bounds (A will be in view center and B will hit a views border)

Color lines are AB vectors. Thin black lines are characters heights. On the right is the 2D screen

1. Maybe add a rule, that everything behind camera always goes on to bottom edge
2. Draw the marker in screen space with regard to markers sizes and overlaps

You might need to try that to see if marker going off viewport will jump between positions. If that happens you can try lerping the position when marker approaches the border of the viewport.

• How exactly would that step 4 work? – Panda Pajama May 14 '15 at 5:36
• @PandaPajama: I have added a little to the answer. Is it more clear now? – Kromster May 14 '15 at 5:53
• Not really. How are you "converting and normalizing" the vector? If that is by applying a projection matrix, then you're doing the same thing I'm doing – Panda Pajama May 14 '15 at 6:02
• @PandaPajama: As far as I understand you are doing it in camera-space (hence the negative Z W problem). I suggest to do it in world space and only then convert to screen-space. – Kromster May 14 '15 at 6:43
• But, by calculating AB, am I not converting the target position from world coordinates to camera coordinates? – Panda Pajama May 14 '15 at 7:07

Given the three vectors [camera position], [camera direction] and [object position]. Compute the dot product of [camera direction] and [object position]-[camera position]. If the result is negative, the object is behind the camera.

Given that I understood your code right it would be:

if((PositionX - Camera.PositionX) * Camera.LookAtX
+ (PositionY - Camera.PositionY) * Camera.LookAtY
+ (PositionZ - Camera.PositionZ) * Camera.LookAtZ
< 0)

• Y is PositionY + (y * Scaling). I'll give it a try tonight – Panda Pajama May 14 '15 at 6:36
• With this I can indeed know when the point is behind the camera. However, this doesn't prevent the projection to reach a singularity at Z=0. – Panda Pajama May 19 '15 at 11:55
• @PandaPajama Is that a practical issue? The probability of Z being exactly 0 should be very small, most players will never experience it, and the gameplay impact in the expected few cases is a negligible single frame visual glitch. I'd say your time is better spent elsewhere. – aaaaaaaaaaaa May 19 '15 at 15:05

Test only (projected.W < 0), not (Z < 0 || W < 0).

In some clip space formulations (cough OpenGL cough), the visible range includes positive and negative Z values, whereas W is always positive in front of the camera and negative behind.

• I've tried with only W < 0, but there are still places where it is incorrectly projected. – Panda Pajama May 21 '15 at 6:53
float screenX = (((projectedVector.X / abs(projectedVector.W)) + 1.0f) / 2.0f) * viewportWidth;
float screenY = ((1.0f - (projectedVector.Y / abs(projectedVector.W))) / 2.0f) * viewportHeight;


And remove this part

// Invert X and Y when behind the camera
if (projectedVector.Z < 0 || projectedVector.W < 0)
{
screenX = -screenX;
screenY = -screenY;
}


Try a billboard, which will automatically make the pic face the camera, and will automatically draw the pic onto the screen at the proper location

• Have you read the question? – Kromster Jun 1 '15 at 13:10
• Sorry, let me explain - It is possible to take the output of a billboard, get rid of the scaling, (so it's 2D but follows the 3d position), and then keep it in the bounds of the screen using some matrix math. – alaokastrail Aug 15 '15 at 19:05
• The question is specifically about the "and then keep it in the bounds of the screen using some matrix math" – Kromster Aug 16 '15 at 8:38