I am trying to use my own collision detection system in Unity to speed up my game. My problem is that when rigid bodies sit on top of static bodies, they will balance unrealistically on one edge or a corner. I really just need a better way of fixing this issue, but I will explain further.
The way I am detecting collisions right now will probably change because it isn't great but I am simply making a list of unique vertices of the mesh, and testing if those points are inside another mesh (it's a voxel game so that is a bit easier than it might otherwise be). I eventually want to either figure out a better way or just cast points along the edges too and call it good enough.
I have tried adding a force to the point of collision to try and "settle" the rigid body into place, and this sort of works, it's just that once it has settled, it vibrates wildly just sitting on top of a static mesh... I can get it to either slowly float down into a settled position or it does it fast but vibrates wildly after settling on a static mesh.
I would appreciate any ideas at all, even if they are bad ones. Thank you.