Many questions have been posted on this topic but I'm facing a weird behavior which I could not find an explanation for.

I perform a raycast giving it a LayerMask parameter which is defined as a public variable and set in the inspector with the layers "Wall" and "Enemy" ticked.

At first glance it appeared to work fine because the raycast rightly stops on GameObjects with that layer.

But then I saw that it also stopped on GameObjects set to the default layer, as if the LayerMask was ignored. Is there something I'm missing?

According to this answer I have set up everything correctly and I expected it to work fine.

Looking at the inspector at runtime I can clearly see that my mask is set only to "Wall" and "Enemy", and that the GameObject the collider is attached to has default layer set. I'm then stating that the mask the way I'm using it does not exclude other layers. The only way I can get this working is by setting that GameObject layer to Ignore Raycast, but in this case the LayerMask is not necessary.

I'm calling the function this way:

RaycastHit2D inSight = Physics2D.Raycast(start, end, lm); 

Maybe is there something different with Physics2D and raycasts?
I would really prefer to use LayerMasks instead of specifying Ignore Raycast on whatever I want to ignore. Any clue?


1 Answer 1


I've had this happen with the 3D raycast in the past, and it seems to be because the raycast functions have so darned many optional parameters, and LayerMasks aren't picky about what type they're treated as.

From the docs:

public static RaycastHit2D Raycast(
       Vector2 origin, 
       Vector2 direction, 
       float distance = Mathf.Infinity, 
       int layerMask = DefaultRaycastLayers, 
       float minDepth = -Mathf.Infinity, 
       float maxDepth = Mathf.Infinity

layerMask is the fourth parameter. So if you provide only three parameters, Unity wants to interpret them as origin, direction, and distance.

It might seem like providing an input of type LayerMask should make your intent unambiguous, or at least generate a compile-time error, but LayerMask can implicitly convert to int, and int can implicitly convert to float.

So when you put a LayerMask in the third argument, the compiler tries to convert it to a float, succeeds, and so silently re-interprets your mask as a distance. The fourth parameter layerMask is then unspecified, so it defaults to DefaultRaycastLayers, basically ignoring the layers you chose in the Inspector.

You can fix this by providing a dummy distance value (say, infinity, if you don't have any more conservative upper bound) so your layer mask selections end up in the fourth argument where Unity expects it:

RaycastHit2D inSight = Physics2D.Raycast(start, end, float.PositiveInfinity, lm);

  • \$\begingroup\$ Thank you gregory but I think this is wrong. For a simple reason which is that layerMask in my raycast function is a LayerMask and cannot be mismatched for a float. Simply unity has overload for Raycasy(vector, vector, LayerMask) \$\endgroup\$
    – Leggy7
    Commented Oct 15, 2015 at 8:10
  • 1
    \$\begingroup\$ I've explained how the LayerMask type can in fact be mismatched for a float. Try it — the compiler might not be using the overload that you think it should. I've had this happen in my own code too. \$\endgroup\$
    – DMGregory
    Commented Oct 15, 2015 at 12:09
  • \$\begingroup\$ Also, @Leggy7, I can't find the overload that you describe. In both Unity 4 & 5, Physics2D.Raycast reports only the one definition, with its various optional parameters, and distance always comes before layerMask. Physics.Raycast (the 3D version) does have an overload that takes (ray, distance, layerMask) which is the only 3-argument version I can find containing layerMask, but even there distance comes first. \$\endgroup\$
    – DMGregory
    Commented Oct 15, 2015 at 13:51
  • 1
    \$\begingroup\$ Now, at home, I tried it. Shame on me, it works perfectly. I got fooled by the fact that no compilers errors occurred giving overload with 3rd parameter as LAyerMask insted of float: I assumed such overload existed. Instead, apparently, unity can convert a LayerMask to a float implicitly. Thank you Gregory,your answer is correct! \$\endgroup\$
    – Leggy7
    Commented Oct 15, 2015 at 20:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .