In 2D, how can I improve character rotation using raycasts when dealing with sharp/convex edges?

Edit:

After making some edits to my polygon collider (based on @Alex F's suggestion), I can indeed confirm that sharp, convex corners are causing the buggy rotation. For now I've decided to only rotate my character if its moving, so standing on a corner when velocity.x = 0 won't cause my collider to rotate rapidly. However, this seems like a band-aid fix, and I'm still wondering if there's a better way to handle these edges. Resuming movement after idling on a corner still causes some buggy behavior, and I'm concerned that this will compound once my rotations become more complicated or frequent:

Original question:

I'm quite inexperienced with quaternions and rotating, so I'm not entirely sure if this issue has to do with my code, the collisions shown in this question, or my game design.

What I'm trying to achieve is "smooth" rotation of my character's collider based on the normal of the current surface. I've been creating a custom raycast controller in 2D from the ground up, and already know how to find all kinds of information from rays, such as the current angle of a sloped surface. My controller is coming along quite nicely, and rotation is one of the last things I'd like to tackle. It should be able to handle sloped box colliders, but also irregular ground shapes that require something like a polygon collider.

I've been somewhat successful, having learned that I shouldn't set my character's z-rotation directly, but instead update my entire transform.rotation with a method like Quaternion.Lerp. This has worked generally well, but I'm running into issues on irregular ground, and on corners.

Notice the smooth transition from flat to sloped surfaces, but that my collider rotates rapidly when the raycast hits a corner:

Movement is still somewhat smooth on irregular terrain, but there is a noticeable shakiness at times. Sometimes my character's collider rotates rapidly as well (end of gif):

I suspect that the shakiness on the irregular terrain is just my collider getting "caught" on the many edges that the polygon collider creates. I just don't know if this is happening because I could be doing something differently in my code, or if I need to edit the polygon collider, or is caused by something else entirely. Perhaps raycasting isn't the best way to achieve my desired behavior.

[SerializeField]
private float speed = 5;
[SerializeField]
private float gravity = -20;
[SerializeField]

private BoxCollider2D boxCollider;
private Vector2 velocity;
private Vector2 groundNormal;

private void Awake()
{
boxCollider = GetComponent<BoxCollider2D>();
}

private void Update()
{
float playerInput = Input.GetAxisRaw("Horizontal");

velocity.x = playerInput * speed;
velocity.y += gravity * Time.deltaTime;

MoveCharacter(velocity * Time.deltaTime);
}

private void MoveCharacter(Vector2 velocity)
{
//Rotate the character
Quaternion targetRotation = Quaternion.FromToRotation(Vector2.up, groundNormal);
transform.rotation = Quaternion.Lerp(transform.rotation, targetRotation, 0.5f);

groundNormal = Vector2.zero;

CollideBelow(ref velocity);
transform.Translate(velocity);
}

private void CollideBelow(ref Vector2 deltaMovement)
{
float rayLength = 1;
Vector2 rayOrigin = transform.position;

RaycastHit2D hit = Physics2D.Raycast(rayOrigin, -transform.up, rayLength, collisionMask);
Debug.DrawRay(rayOrigin, -transform.up * rayLength, Color.red);

if (hit)
{
velocity.y = 0;
deltaMovement.y = -(hit.distance - boxCollider.size.y * 0.5f);

groundNormal = hit.normal;
}
}

• I don't have the time to flesh this out into a full answer, but you could try to use two raycasts (one on each end of your rectangle), and calculating where the intersection of both hit lines lies. You could then interpolate the rotation of your rectangle based on the distance the intersection has to the middle of your vertex. (Intersection on the far left = rectangle collides fully with the right one, and vice versa.) – Peethor Mar 25 '19 at 10:13

I don't know if raycasting is the best way to do what you want, but it certainly looks like it works. I think the problem is just lack of detail on the track; if you rounded the sharp corners of your track, it looks like it would work perfectly.

• I'm not sure if you can reply to my comment, but do you know of a decent way to "round" the edges of a polygon collider? I've been playing with the collider editor for quite some time tonight, but haven't been able to eliminate problem areas with enough certainty. – Ginger and Lavender Feb 23 '19 at 7:55
• I've never used the collider editor, but maybe using a spline would work. Again, never used the editor, so I can't help much here. Also, yeah, I can reply to comments, but not add new ones since I don't have 50 rep yet. – Alex F Feb 23 '19 at 8:11

You can add a circle2D collider as a trigger (and just make sure it's slightly bigger than the square), then do something like this to get the groundNormal:

// cache reference to the circle2D trigger in Awake. I'll call it trigger

void OnTriggerStay2D(Collider2D other) {
groundNormal = -trigger.Distance(other).normal;
}



I haven't tested how this would work in your case, but I'm using literally the above code for a circular character, and groundNormal transitions very smoothly over corners, so I think it should work for you.

I think on the edge, it takes much more than one raycast (example: 1 bottom left, 1 bottom right ...), because the raycast hesitates to give you between two normal ground

ps: sorry for my english

• This answer would be better if it explained how to combine the results of more than one raycast to get a better outcome. – DMGregory Jun 1 '19 at 15:09