How to determine if an object is within field of view given xyz of object, player position, and 3D vector for player view?

Question

I only have the object's position, the player position, and the vector representing the player's view, and I have the FoV (120 degrees horizontal/60 on each side, and 60 degrees vertical/30 on each side).

As I'm dealing with data that has already been recorded, I cannot use any Unity built-in functions and have to calculate it manually. My thoughts were to convert the 3D coordinates into 2D vectors, so that I can calculate if an object is in the horizontal plane first (by taking the X and Z values) and then if it is, check if it is in the vertical plane (take the Y and Z coordinates).

After, I calculate the the angle between the object and the player view to see if the absolute value of that is within the field of view (for example, if the object is on the left side of the player view, and the horizontal FOV is 120 degrees, then if the absolute value of the angle is less than or equal to 60 degrees it is within view).

Is this on the right track? I've been getting strange results (like only one of the four objects I'm looking for shows as visible throughout a log of values even though they should all at some point be visible).

Here is some code to show what I'm doing:

    public static Double angle(Vector2D obj, Vector2D bodyPos, Vector2D headDir) {

    Double result = 0.0;

    // Get the vector between the body position and the object
    Vector2D temp = Vector2D.subtract(obj, bodyPos);

    // First find dot product
    Double dotProduct = Vector2D.dotProduct(headDir, temp);

    // Now find the length of each vector and get the product of them
    Double lengthProduct = Vector2D.vectorLength(headDir)*Vector2D.vectorLength(temp);

    // Find the intermediary result
    result = dotProduct / lengthProduct;

    // Apply arc cos to find the angle between the vectors
    result = Math.acos(result);

    // Convert angle to degrees
    result = (result * 180)/Math.PI;
    System.out.println("The angle is: " + result);

    return result;
}

And here is the code that determines if something is in view by using that angle function:

    public static boolean isInView(bodyPosition objPos, bodyPosition bodyPos, headDirection headDir){

    boolean inView = false;

    Double h_angle = 0.0;
    Double v_angle = 0.0;

    // Create 2D vectors for both the horizontal and vertical FOV measurements
    Vector2D h_objPos = new Vector2D(objPos.getDirectionX(), objPos.getDirectionZ());
    Vector2D v_objPos = new Vector2D(objPos.getDirectionY(), objPos.getDirectionZ());

    Vector2D h_bodyPos = new Vector2D(bodyPos.getDirectionX(), bodyPos.getDirectionZ());
    Vector2D v_bodyPos = new Vector2D(bodyPos.getDirectionY(), bodyPos.getDirectionZ());

    Vector2D h_headDir = new Vector2D(headDir.getDirectionX(), headDir.getDirectionZ());
    Vector2D v_headDir = new Vector2D(headDir.getDirectionY(), headDir.getDirectionZ());

    // determine angle of FOV+
    Double iHFOV = FOV_H_MID * 0.5;
    Double iVFOV = FOV_V_MID * 0.5;

    // Check horizontal FOV first; if it is within that, then check the vertical
    // If both pass, then it is in view
    h_angle = Vector2D.angle(h_objPos, h_bodyPos, h_headDir);
    v_angle = Vector2D.angle(v_objPos, v_bodyPos, v_headDir);
    if ( Math.abs( h_angle ) <= iHFOV) {
        if ( Math.abs( v_angle ) <= iVFOV) {
            inView = true;
        }
    }

    return inView;
}

I found some resources that I will be reviewing in the mean time, but any help is appreciated. I haven't done any linear algebra courses so I'm trying to wrap my brain around it.

Answer 1

To be honest, you're best to test using bounding boxes and the camera frustum if you can get hold of those.

I've attached my code for generating the frustum. You then should test each corner of your bounding box against this frustum (code below).

  [MethodImpl(MethodImplOptions.AggressiveInlining)]
        void GenerateFrustumPlanes()
        {
            if (m_disableFrustrumUpdates)
                return;

            Matrix matrix;

            // Create the frustum matrix from the view matrix and updated projection matrix.
            matrix = m_wvp;

            m_planes[0] = new Vector4(matrix.M13,
                 matrix.M23,
                 matrix.M33,
                matrix.M43); // near
            m_planes[1] = new Vector4(matrix.M14 - matrix.M13,
                matrix.M24 - matrix.M23,
                matrix.M34 - matrix.M33,
                matrix.M44 - matrix.M43);

            m_planes[2] = new Vector4(matrix.M14 + matrix.M11,
                matrix.M24 + matrix.M21,
                matrix.M34 + matrix.M31,
                matrix.M44 + matrix.M41); // left
            m_planes[3] = new Vector4(matrix.M14 - matrix.M11,
                matrix.M24 - matrix.M21,
                matrix.M34 - matrix.M31,
                matrix.M44 - matrix.M41);//right

            m_planes[4] = new Vector4(matrix.M14 - matrix.M12,
                matrix.M24 - matrix.M22,
                matrix.M34 - matrix.M32,
                matrix.M44 - matrix.M42); // top
            m_planes[5]  = new Vector4(matrix.M14 + matrix.M12, matrix.M24 + matrix.M22, matrix.M34 + matrix.M32, matrix.M44 + matrix.M42); // bottom

            for (int i = 0; i < s_frustrumPlanes; i++)
            {
                m_planes[i].Normalize();
            }          
        }

code to test the points of the bbox against the frustrum.

public bool IsInViewableArea(List a_cornerLIst) { // this is a simpler occlusion test where we basically don't even need to test frustum as the the extents are outside the camera view distance. But you need to test extents + width because we only hold 2 corners and squares are 4

    int iTotalIn = 0;

    Vector4[] pList = GetFrustumPlanes();  // this should only be called once.

    // test all 8 corners against the 6 sides 
    // if all points are behind 1 specific plane, we are out
    // if we are in with all points, then we are fully in
    Vector4 temp;
    float dotvector;
    for (int p = 0; p < Camera3d.s_frustrumPlanes; ++p)
    {

        int iInCount = 8;
        int iPtIn = 1;

        for (int i = 0; i < a_cornerLIst.Count; ++i)
        {
            //Vector3 a = pList[i].
            // test this point against the planes
            //   if (pList[p].SideOfPlane(vList[i]) == BEHIND)
            //Vector4.Dot()
            temp = (Vector4)(a_cornerLIst[i] );
            temp.W = 1;

            dotvector = Vector4.Dot(pList[p], temp);

            if (dotvector < 0.0f)
            {
                iPtIn = 0;
                --iInCount;
            }
        }

        // were all the points outside of plane p?
        if (iInCount == 0)
        {
            return false;
        }
        // check if they were all on the right side of the plane
        iTotalIn += iPtIn;
    }

    // so if iTotalIn is 6, then all are inside the view
    //    if (iTotalIn == 6)
    //       return true;

    // we must be partly in then otherwise
    return true;
}

Answer 2

Yes, your approach is valid. If z is up, then first do in xy plane, and then check vertical, taking into account that the camera may be tilted up or down.

Note that to do tests in 2D, I simply use 3D vectors, but set z to zero.

Note that it doesn't handle special case of camera looking straight up (0,0,1) or straight down (0,0,-1), which would cause the normalize() to fail.

Note that this vertical test also won't work if camera tilt approaches straight up or down, where tilt + 0.5 vertical fov exceeds 0.5 pi, or tilt - 0.5 vertical fov is below -0.5 pi.

bool inview( vec3 campos, vec3 camdir, bodypos )
{
    // First test if in our horizontal field of view.
    vec3 tobody = bodypos - campos;
    vec3 tobody2d = tobody;
    tobody2d.z = 0;
    vec3 camdir2d = camdir;
    camdir2d.z = 0;
    tobody2d = tobody2d.normalize();
    camdir2d = camdir2d.normalize();
    float dp = tobody2d.dot( camdir2d );
    if ( dp < cosf( horizontal_fov ) )
        return false; // body too far to left or right of camera.
    // Check if in vertical fov (Note that camera may be tilted up/dn.)
    float tiltcam = asinf( camdir.z );
    tobody = tobody.normalize();
    float tilttobody = asinf( tobody.z );
    if ( tilttobody < tiltcam - 0.5 * vertical_fov )
        return false; // body too low to see.
    if ( tilttobody > tiltcam + 0.5 * vertical_fov )
        return false; // body too high to see.
    return true;
}

Answer 3

ErnieDingo's got a pretty good answer, what with using frustum planes. I had a similar problem I wanted to solve a few years back: calculating a field of view as if an object had a camera without needing a rendering Camera component.

So this is pretty similar, but my code is different.

public static Plane[] CalculateFrustum(Vector3 origin, Vector3 direction, float fovRadians, float viewRatio, float distance) {
    Vector3 nearCenter = origin + direction * 0.3f;
    Vector3 farCenter  = origin + direction * distance;
    Vector3 camRight   = Vector3.Cross(direction,Vector3.up) * -1;
    Vector3 camUp      = Vector3.Cross(direction,camRight);

    float nearHeight = 2 * Mathf.Tan(fovRadians / 2) * 0.3f;
    float farHeight  = 2 * Mathf.Tan(fovRadians / 2) * distance;
    float nearWidth  = nearHeight * viewRatio;
    float farWidth   = farHeight * viewRatio;

    Vector3 farTopLeft  = farCenter + camUp*(farHeight*0.5f) - camRight*(farWidth*0.5f);//new Vector3(-camRight.x * (farWidth*0.5f), (farHeight*0.5f), -camRight.z * (farWidth*0.5f));
    //Vector3 farTopRight = farCenter + camUp*(farHeight*0.5f) + camRight*(farWidth*0.5f);//new Vector3( camRight.x * (farWidth*0.5f), (farHeight*0.5f),  camRight.z * (farWidth*0.5f));
    Vector3 farBottomLeft  = farCenter - camUp*(farHeight*0.5f) - camRight*(farWidth*0.5f);//new Vector3(-camRight.x * (farWidth*0.5f), -(farHeight*0.5f), -camRight.z * (farWidth*0.5f));
    Vector3 farBottomRight = farCenter - camUp*(farHeight*0.5f) + camRight*(farWidth*0.5f);//new Vector3( camRight.x * (farWidth*0.5f), -(farHeight*0.5f),  camRight.z * (farWidth*0.5f));

    Vector3 nearTopLeft  = nearCenter + camUp*(nearHeight*0.5f) - camRight*(nearWidth*0.5f);//new Vector3(-camRight.x * (nearWidth*0.5f), farCenter.y + (nearHeight*0.5f), -camRight.z * (nearWidth*0.5f));
    Vector3 nearTopRight = nearCenter + camUp*(nearHeight*0.5f) + camRight*(nearWidth*0.5f);//new Vector3( camRight.x * (nearWidth*0.5f), farCenter.y + (nearHeight*0.5f),  camRight.z * (nearWidth*0.5f));
    //Vector3 nearBottomLeft  = nearCenter - camUp*(nearHeight*0.5f) - camRight*(nearWidth*0.5f);//new Vector3(-camRight.x * (nearWidth*0.5f), farCenter.y - (nearHeight*0.5f), -camRight.z * (nearWidth*0.5f));
    Vector3 nearBottomRight = nearCenter - camUp*(nearHeight*0.5f) + camRight*(nearWidth*0.5f);//new Vector3( camRight.x * (nearWidth*0.5f), farCenter.y - (nearHeight*0.5f),  camRight.z * (nearWidth*0.5f));
    Plane[] planes = {
        new Plane(nearTopLeft,farTopLeft,farBottomLeft),
        new Plane(nearTopRight,nearBottomRight,farBottomRight),

        new Plane(farBottomLeft,farBottomRight,nearBottomRight),
        new Plane(farTopLeft,nearTopLeft,nearTopRight),

        new Plane(nearBottomRight,nearTopRight,nearTopLeft),
        new Plane(farBottomRight,farBottomLeft,farTopLeft)};

    return planes;
}

Which calculated the Plane[] needed for this extension function:

public static class GameObjectExtensions {
    public static bool IsVisibleFrom(this GameObject obj, Plane[] planes) {
        Bounds b = obj.GetComponent<Collider>().bounds;
        b.Expand(0.5f);
        return GeometryUtility.TestPlanesAABB(planes, b);
    }
}

I could now call someGameObject.IsVisibleFrom(...) and pass in the necessary values, including the return from CalculateFrustum() which would even take in a field of view (in Radians), view ratio, and max view distance (I also had an IsVisibleFrom() that took in a Camera object as well, which uses GeometryUtility.CalculateFrustumPlanes(camera); to get its Plane[]).

As far as I remember, the CalculateFrustum() method I have is identical to what CalculateFrustumPlanes() does.