How do I correctly project a point behind the camera?

Question

I am making a 3D game in which I put an exclamation marker above points of interest.

In order to find out where in the 2D screen should I put my marker, I am manually projecting the 3D point where the marker should be.

It looks like this:

Looks pretty good. When the marker is outside of the screen, I simply clip the coordinates so they fit into the screen. It looks like this:

So far the idea is going pretty well. However, when the points of interest are behind the camera, the resulting X,Y coordinates are inverted (as in positive/negative), and I get the marker to appear on the opposite corner of the screen, like this:

(The projected-then-clamped point is the tip of the marker. Don't mind the rotation of the marker)

It makes sense, since the coordinates behind the frustum are inverted in X and Y. So what I'm doing is to invert the coordinates when they are behind the camera. However, I still don't know what is exactly the condition when the coordinates should be inverted.

This is currently what my projection code looks like (in C# with SharpDX):

    public override PointF ProjectPosition(float viewportWidth, float viewportHeight, float y)
    {
        var projectionMatrix = Matrix.PerspectiveFovRH(GetCalibratedFOV(Camera.FOV, viewportWidth, viewportHeight), viewportWidth / viewportHeight, Camera.Near, Camera.Far);
        var viewMatrix = Matrix.LookAtRH(new Vector3(Camera.PositionX, Camera.PositionY, Camera.PositionZ), new Vector3(Camera.LookAtX, Camera.LookAtY, Camera.LookAtZ), Vector3.UnitY);
        var worldMatrix = Matrix.RotationY(Rotation) * Matrix.Scaling(Scaling) * Matrix.Translation(PositionX, PositionY, PositionZ);
        var worldViewProjectionMatrix = worldMatrix * viewMatrix * projectionMatrix;

        Vector4 targetVector = new Vector4(0, y, 0, 1);
        Vector4 projectedVector = Vector4.Transform(targetVector, worldViewProjectionMatrix);

        float screenX = (((projectedVector.X / projectedVector.W) + 1.0f) / 2.0f) * viewportWidth;
        float screenY = ((1.0f - (projectedVector.Y / projectedVector.W)) / 2.0f) * viewportHeight;
        float screenZ = projectedVector.Z / projectedVector.W;

        // Invert X and Y when behind the camera
        if (projectedVector.Z < 0 ||
            projectedVector.W < 0)
        {
            screenX = -screenX;
            screenY = -screenY;
        }

        return new PointF(screenX, screenY);
    }

As you can see, my current idea is to invert the coordinates when either the Z or W coordinates are negative. It works -most- of the time, but there are still some very specific camera locations where it doesn't work. In particular this point shows one coordinate working and the other one not (the correct location should be the bottom right):

I have tried inverting when:

• Z is negative (this is what makes most sense to me)
• W is negative (I don't understand the meaning of a negative W value)
• Either Z or W is negative (what's currently working most of the time)
• Z and W are of different sign, a.k.a.: Z / W < 0 (makes sense to me. doesn't work though)

But have still not found a consistent way with which all my points are correctly projected.

Any ideas?

Answer 1

How about doing it a bit differently?

Start as usual and render all markers within viewport. If a marker is outside of the viewport - proceed:

1. Get positions of the marker A
2. Get position of the camera B (maybe slightly in front of it)
3. Calculate 3D vector AB between these two
4. Convert and normalize that vector into 2D screen bounds (A will be in view center and B will hit a views border)

Color lines are AB vectors. Thin black lines are characters heights. On the right is the 2D screen

5. Maybe add a rule, that everything behind camera always goes on to bottom edge
6. Draw the marker in screen space with regard to markers sizes and overlaps

You might need to try that to see if marker going off viewport will jump between positions. If that happens you can try lerping the position when marker approaches the border of the viewport.

Answer 2

Given the three vectors [camera position], [camera direction] and [object position]. Compute the dot product of [camera direction] and [object position]-[camera position]. If the result is negative, the object is behind the camera.

Given that I understood your code right it would be:

if((PositionX - Camera.PositionX) * Camera.LookAtX
    + (PositionY - Camera.PositionY) * Camera.LookAtY
    + (PositionZ - Camera.PositionZ) * Camera.LookAtZ
    < 0)

Answer 3

Test only (projected.W < 0), not (Z < 0 || W < 0).

In some clip space formulations (cough OpenGL cough), the visible range includes positive and negative Z values, whereas W is always positive in front of the camera and negative behind.

Answer 4

float screenX = (((projectedVector.X / abs(projectedVector.W)) + 1.0f) / 2.0f) * viewportWidth;
float screenY = ((1.0f - (projectedVector.Y / abs(projectedVector.W))) / 2.0f) * viewportHeight;

And remove this part

// Invert X and Y when behind the camera
if (projectedVector.Z < 0 || projectedVector.W < 0)
{
    screenX = -screenX;
    screenY = -screenY;
}

Answer 5

Try a billboard, which will automatically make the pic face the camera, and will automatically draw the pic onto the screen at the proper location