Does making use of this setting decrease performance, or increase
It (generally) increases performance. If you only have a handful of colliders in a scene, this could result in a slight performance hit, but it wouldn't be noticeable. Best practice is if you don't ever intend two layers to interact with each other, go ahead and disable it in the Matrix.
I could imagine it decreases it if every potential collision had to be
checked for it's layer. Or are those collisions completely bypassed?
Your assumptions are correct.
The Matrix is a filter on objects: If the object currently being checked belongs to this group, then check for collisions against object belonging to checked groups.
This means, for layers where you've disabled some collisions in the matrix, you are only checking for collisions between subsets of the colliders in the scene. Since there's less checks to perform, you gain some performance.
This is really only necessary as a performance optimization when you have a large number of objects in a scene. Most of the time this is used to support game functionality. For instance if you want bullets to only impact with players and walls, but not balloons that get pushed around the room by players as they move.