Raycast2D hits acting (seemingly) inconsistently

Question

I'm using Unity to make a 2D game, and I'm trying to put together a vision cone/flashlight system that's a lot less of a headache than the systems that are currently out there.

Despite this, I'm still slamming my head into a wall when it comes to these raycasts. See this video.

Here's the vision cone script:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class VisionCone : MonoBehaviour
{
    public List<GameObject> targetsInView = new List<GameObject>();
    public Collider2D[] collidersInRange;
    public float viewDistance;
    [Range (0, 360)]
    public float viewAngle;
    public GameObject forwardAnchor;

    private int targetMask;
    private int wallMask;
    private Vector3 origin;
    private Vector3 lightDir;

    void Start()
    {
        targetMask = 1 << 9;
        wallMask = 1 << 8;
    }

    void Update()
    {
        origin = transform.position;
        Vector3 heading = forwardAnchor.transform.position - origin;
        float anchorDistance = heading.magnitude;
        lightDir = heading / anchorDistance;

        collidersInRange = Physics2D.OverlapCircleAll(origin, viewDistance, targetMask);

        foreach (Collider2D collider in collidersInRange)
        {
            Vector3 colliderPos = new Vector3(collider.transform.position.x, collider.transform.position.y, 0);
            Vector3 targetDir = colliderPos - origin;

            Debug.DrawLine(origin, colliderPos, Color.white);

            if (Vector3.Angle(targetDir, lightDir) < viewAngle / 2)
            {
                List<Transform> vertices = new List<Transform>();
                Transform[] allChildren = collider.gameObject.GetComponentsInChildren<Transform>();

                foreach (Transform child in allChildren)
                {
                    if (child.gameObject.tag == "Vertex")
                    {
                        vertices.Add(child);
                    }
                }

                foreach (Transform vertex in vertices)
                {
                    float vertDistance = Vector3.Distance(origin, vertex.position);
                    RaycastHit2D hit = Physics2D.Raycast(origin, vertex.position, vertDistance, wallMask);

                    if (!hit)
                    {
                        Debug.DrawLine(origin, vertex.position, Color.green);
                        if (!targetsInView.Contains(collider.gameObject))
                        {
                            targetsInView.Add(collider.gameObject);
                        }
                    }
                    else
                    {
                        Debug.DrawLine(origin, vertex.position, Color.yellow);
                        if (targetsInView.Contains(collider.gameObject))
                        {
                            targetsInView.Remove(collider.gameObject);
                        }
                    }
                }
            }
            else
            {
                Debug.DrawLine(origin, colliderPos, Color.red);
                if (targetsInView.Contains(collider.gameObject))
                {
                    targetsInView.Remove(collider.gameObject);
                }
            }

        }
    }

    void LateUpdate()
    {
        foreach (GameObject target in targetsInView)
        {
            float targetDistance = Vector3.Distance(target.transform.position, origin);
            if (targetDistance > viewDistance + 0.5)
            {
                targetsInView.Remove(target);
            }
        }
    }
}

As you can see, the wall is inconsistent in blocking the raycast - sometimes every single ray should collide with the wall and therefore turn yellow, but they're all green. Sometimes they're yellow even when they don't collide with a wall. Sometimes it's a mix, and so on.

I'm at my wit's end with these - I tried the same method using the vertices of a box collider 2D which I found using a foreach collider loop, to the same effect.

I have absolutely no idea why the hit detection is so inconsistent. Any help would be greatly appreciated.

Answer 1

Here's your problem:

float vertDistance = Vector3.Distance(origin, vertex.position);
RaycastHit2D hit = Physics2D.Raycast(origin, vertex.position, vertDistance, wallMask);

If we read the documentation for Physics2D.Raycast, we see the signature described like this:

public static RaycastHit2D Raycast(Vector2 origin, Vector2 direction, float distance = Mathf.Infinity, int layerMask = DefaultRaycastLayers, float minDepth = -Mathf.Infinity, float maxDepth = Mathf.Infinity);

Notice the second parameter is a direction:

direction  A vector representing the direction of the ray.

But here you've passed the position of the vertex you want to shoot toward. This matches the direction only if our origin is on the line between this point and (0, 0, 0). If our origin is anywhere else in the world, then we end up firing the ray in a direction that has nothing to do with where we're trying to aim.

You could convert your position to a direction by subtracting your starting point:

Vector2 offset = (Vector2)(vertex.position - origin);

...but it looks like what you want here is a LineCast, which is a point-to-point raycast, and will let you skip calculating your vertDistance at all:

RaycastHit2D hit = Physics2D.LineCast(origin, vertex.position, wallMask);