Camera Jitters When Displacing and Rotating Smoothly

Question

DISCLAIMER: I am using Unity 5.6.5f1 for this code, I honestly don't know if there would be API updates on later versions, though I don't think so.

I tried to make a simple Third Person Camera that displaces and rotates towards a given target, with the following key parameters:

• Following types for both displacement and rotation. Right now the enum is just { Instant, Smooth }.
• Axes to ignore for both displacement and rotation, all axes in an enum.
• Axes to invert for the orbiting functionality.
• 'Relative Following' for both displacement and rotation. If the flags are checked the displacement/rotation will be relative to the target's orientation ('target.rotation * displacement' and 'target.rotation * rotation' for displacement and rotation respectively).
• An offset vector, which is affected by the displacement relative following's flag. It is treated as a normalized vector at runtime.
• A scalar of the aforementioned offset. Which is basically de distance between the camera and the target.
• An offset vector as Euler for the rotation, it is equally affected by the rotation relative following's flag.
• Other attributes, such as 'displacementFollowDuration', 'maxDisplacementFollowSpeed', 'rotationFollowDuration', 'maxRotationFollowingSpeed', etc., are for the smooth following. The attributes not mentioned are either self-explanatory or irrelevant (at least that's what I want to think, correct me if I may be wrong).

The Problem:

When the smooth flags are enabled for both displacement and rotation, the camera starts to shake, I think it has something to do with the rotation.

Things I've tried already:

• Call the rotation and displacement following on different threads (being Update, Late Update and FixedUpdate), not yet on coroutines. This time I made an enum 'LoopType' to encapsulate all threads, and avoid re-compiling each time I move the functions.
• Use different kinds of time deltas. By the same fashion of the threads, I made an enum 'TimeDelta' which encapsulates the 3 types of time deltas Unity offers.
• Change the smooth functions for both displacement and rotation, such as Lerp, SmoothDamp, Slerp, even custom elastic functions, etc.
• Sacrifing 1 frame by following the target's point of the last frame, as proposed here.
• Given that the target's displacement is that of a Rigidbody, this post suggests to attach a Rigidbody to the camera and displace/rotate it by Rigidbody.position/Rigidbody.rotation respectively.

Camera's Script:

using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
#if UNITY_EDITOR
using UnityEditor;
#endif

[Flags]
public enum Axes
{
    None = 0,
    X = 1,
    Y = 2,
    Z = 4,

    XAndY = X | Y,
    XAndZ = X | Z,
    YAndZ = Y | Z,
    All = X | Y | Z
}

public enum LoopType { Update, LateUpdate, FixedUpdate }

public enum TimeDelta { Default, Fixed, Smooth }

public enum FollowingType { Instant, Smooth }

/// \TODO Maybe something like Lucky Tale and Resident Evil 4 and 5 camera (return to the original rotation if there are no axes)
public class GameplayCamera : MonoBehaviour
{
    public Transform target;                        /// Camera's Target.
    [Space(5f)]
    [Header("Displacement Following's Attributes:")]
    public LoopType followDisplacementAt;           /// Loop to do the Displacement Following.
    public FollowingType displacementFollowType;    /// Type of Following for Displacement.
    public TimeDelta displacementTimeDelta;         /// Displacement's Time Delta.
    public Axes ignoreDisplacementAxes;             /// Displacement Axes to Ignore.
    public Axes invertAxes;                         /// Axes to Invert.
    public Axes limitOrbitAxes;                     /// Orbit's Axes to Limit.
    public bool relativeDisplacementFollow;         /// Follow Target's Displacement Relative to Target's Orientation?.
    public bool limitDisplacementFollow;            /// Limit Displacement Following's Speed?.
    public Vector3 displacementOffset;              /// [Normalized] Displacement Offset between Camera and Target.
    public Vector2 orbitSpeed;                      /// Orbit's Speed on each Axis.
    public Vector2 minOrbitLimits;                  /// Orbit's Negative Boundaries.
    public Vector2 maxOrbitLimits;                  /// Orbit's Positive Boundaries.
    public float displacementFollowDuration;        /// Displacement's Follow Duration.
    public float maxDisplacementFolowSpeed;         /// Maximum Displacement's Follow Duration.
    public float minDistance;                       /// Minimum Distance Between Camera and Target.
    public float maxDistance;                       /// Maximum Distance Between Camera and Target.
    [Space(5f)]
    [Header("Rotation Following's Attributes:")]
    public LoopType followRotationAt;               /// Loop to do the Rotation Following.
    public FollowingType rotationFollowType;        /// Type of Following for Rotation.
    public TimeDelta rotationTimeDelta;             /// Rotations' Time Delta.
    public Axes ignoreRotationAxes;                 /// Rotation Axes to Ignore.
    public bool relativeRotationFollow;             /// Follow Target's Rotation Relative to Target's Orientation?.
    public bool limitRotationFollow;                /// Limit Rotation Following's Speed?.
    public Vector3 eulerRotationOffset;             /// Rotation Offset between Camera and Target as Euler.
    public float rotationFollowDuration;            /// Rotation's Following Duration.
    public float maxRotationFollowSpeed;            /// Maximum Rotation's Following Speed.
    [Space(5f)]
    public Vector3 up;                              /// Up Vector's Reference.
    [HideInInspector] public Vector3 forward;       /// Reoriented Forward's Vector.
    private Vector3 eulerOrbitRotation;             /// Current Orbit Rotation as Euler.
    private Vector3 displacementVelocity;           /// Displacement's Velocity.
    private Quaternion orbitRotation;               /// Orbit Rotation as Quaternion.
    private Quaternion rotationOffset;              /// Rotation's Offset as Quaternion.
    private Vector2 inputAxes;                      /// Input's Axes.
    private float currentDistance;                  /// Current Distance from Camera and Player.
    private float angularSpeed;                     /// Angular's Speed.

#region UnityMethods:
    /// <summary>Draws Gizmos on Editor mode.</summary>
    private void OnDrawGizmos()
    {
        Gizmos.color = Color.green;
        Gizmos.DrawRay(transform.position, up);
        Gizmos.color = Color.blue;
        Gizmos.DrawRay(transform.position, forward);

        if(target != null)
        {
            Gizmos.color = Color.cyan;
            Gizmos.DrawLine(target.position, GetOffsetPoint());

            if(!Application.isPlaying)
            {
                UpdateRotationOffset();
                ReorientForward();
            }

            Quaternion rotation = transform.rotation * rotationOffset;

            Handles.color = new Color(1.0f, 0.0f, 0.0f, 0.35f); /// Red
            Handles.DrawSolidArc(transform.position, transform.right, transform.forward, Vector3.Angle(transform.forward, rotation * Vector3.forward) * Mathf.Sign(eulerRotationOffset.x), 1.0f);
            Handles.color = new Color(0.0f, 1.0f, 0.0f, 0.35f); /// Green
            Handles.DrawSolidArc(transform.position, transform.up, transform.right, Vector3.Angle(transform.right, rotation * Vector3.right) * Mathf.Sign(eulerRotationOffset.y), 1.0f);
            Handles.color = new Color(1.0f, 0.0f, 1.0f, 0.35f); /// Blue
            Handles.DrawSolidArc(transform.position, transform.forward, transform.up, Vector3.Angle(transform.up, rotation * Vector3.up) * Mathf.Sign(eulerRotationOffset.z), 1.0f);

            if(!Application.isPlaying)
            {
                transform.position = GetOffsetPoint();
                transform.rotation = Quaternion.LookRotation(GetLookDirection()) * rotationOffset;
            }
        }       
    }

    /// <summary>Resets GameplayCamera's instance to its default values.</summary>
    private void Reset()
    {
        up = Vector3.up;
    }

    /// <summary>GameplayCamera's tick at each frame.</summary>
    private void Update()
    {
        if(target == null) return;

        TrackInput();
        UpdateRotationOffset();

        if(followDisplacementAt == LoopType.Update) DisplacementFollow();
        if(followRotationAt == LoopType.Update) RotationFollow();
    }

    /// <summary>Updates GameplayCamera's instance at the end of each frame.</summary>
    private void LateUpdate()
    {
        if(target == null) return;

        if(followDisplacementAt == LoopType.LateUpdate) DisplacementFollow();
        if(followRotationAt == LoopType.LateUpdate) RotationFollow();

        ReorientForward();
    }

    /// <summary>Updates GameplayCamera's instance at each Physics Thread's frame.</summary>
    private void FixedUpdate()
    {
        if(target == null) return;

        if(followDisplacementAt == LoopType.FixedUpdate) DisplacementFollow();
        if(followRotationAt == LoopType.FixedUpdate) RotationFollow();
    }
#endregion

    /// <summary>Tracks Input.</summary>
    private void TrackInput()
    {
        inputAxes.x = Input.GetAxis("Mouse Y");
        inputAxes.y = Input.GetAxis("Mouse X");
    }

    /// <summary>Performs the Displacement's Following.</summary>
    private void DisplacementFollow()
    {
        if(inputAxes.sqrMagnitude > 0.0f) OrbitInAxes(inputAxes.x, inputAxes.y);

        switch(displacementFollowType)
        {
            case FollowingType.Instant:
            transform.position = GetOffsetPoint();
            break;

            case FollowingType.Smooth:
            transform.position = GetSmoothDisplacementFollowDirection();
            break;
        }
    }

    /// <summary>Performs the Rotation's Following.</summary>
    private void RotationFollow()
    {
        switch(rotationFollowType)
        {
            case FollowingType.Instant:
            transform.rotation = Quaternion.LookRotation(GetLookDirection()) * rotationOffset;
            break;

            case FollowingType.Smooth:
            transform.rotation = GetSmoothFollowRotation();
            break;
        }
    }

    /// <summary>Orbits Camera in Given Axes.</summary>
    /// <param name="x">X's Axis.</param>
    /// <param name="y">Y's Axis.</param>
    private void OrbitInAxes(float x, float y)
    {
        if((invertAxes | Axes.X) == invertAxes) x *= -1.0f;
        if((invertAxes | Axes.Y) == invertAxes) y *= -1.0f;

        float xRotation = (x * orbitSpeed.x * GetTimeDelta(displacementTimeDelta));
        float yRotation = (y * orbitSpeed.y * GetTimeDelta(displacementTimeDelta));

        eulerOrbitRotation.x = (limitOrbitAxes | Axes.X) == limitOrbitAxes ?
            Mathf.Clamp(eulerOrbitRotation.x + xRotation, minOrbitLimits.x, maxOrbitLimits.x) : eulerOrbitRotation.x + xRotation;
        eulerOrbitRotation.y = (limitOrbitAxes | Axes.Y) == limitOrbitAxes ?
            Mathf.Clamp(eulerOrbitRotation.y + yRotation, minOrbitLimits.y, maxOrbitLimits.y) : eulerOrbitRotation.y + yRotation;

        orbitRotation = Quaternion.Euler(eulerOrbitRotation);
    }

    /// <returns>Gets the smooth displacement following's Vector.</returns>
    private Vector3 GetSmoothDisplacementFollowDirection()
    {
        return Vector3.SmoothDamp
        (
            transform.position,
            GetOffsetPoint(),
            ref displacementVelocity,
            displacementFollowDuration,
            limitDisplacementFollow ? maxDisplacementFolowSpeed : Mathf.Infinity,
            GetTimeDelta(displacementTimeDelta)
        );
    }

    /// <summary>Gets Offset Point, with the Orbit's Rotation already combined.</summary>
    private Vector3 GetOffsetPoint()
    {
        Vector3 scaledOffset = displacementOffset.normalized * maxDistance;
        Vector3 point = target.position + (orbitRotation * (relativeDisplacementFollow ? target.rotation * scaledOffset : scaledOffset));

        if((ignoreDisplacementAxes | Axes.X) == ignoreDisplacementAxes) point.x = transform.position.x;
        if((ignoreDisplacementAxes | Axes.Y) == ignoreDisplacementAxes) point.y = transform.position.y;

        return point;
    }

    /// <returns>Looking Direction, taking into account the axes to ignore.</returns>
    private Vector3 GetLookDirection()
    {
        Vector3 direction = target.position - transform.position;

        if((ignoreRotationAxes | Axes.X) == ignoreRotationAxes) direction.x = transform.position.x;
        if((ignoreRotationAxes | Axes.Y) == ignoreRotationAxes) direction.y = transform.position.y;
        if((ignoreRotationAxes | Axes.Z) == ignoreRotationAxes) direction.z = transform.position.z;

        return direction;
    }

    /// <return>Following Rotation, with the Rotation's Offset already combined.</return>
    private Quaternion GetSmoothFollowRotation()
    {
        Quaternion rotation = Quaternion.LookRotation(GetLookDirection()) * rotationOffset;
        float angle = Quaternion.Angle(transform.rotation, rotation);

        if(angle > 0.0f)
        {
            float t = Mathf.SmoothDampAngle(
                angle,
                0.0f,
                ref angularSpeed,
                rotationFollowDuration,
                limitRotationFollow ? maxRotationFollowSpeed : Mathf.Infinity,
                GetTimeDelta(rotationTimeDelta)
            );
            return Quaternion.Slerp(transform.rotation, rotation, t);
        }

        return rotation;
    }

    /// <summary>Updates the Rotation's Offset Given the Wuler Representation.</summary>
    private void UpdateRotationOffset()
    {
        Quaternion rotation = Quaternion.Euler(eulerRotationOffset);
        rotationOffset = relativeRotationFollow ? target.rotation * rotation : rotation;
    }

    /// <summary>Reorients Forward's Vector.</summary>
    private void ReorientForward()
    {
        forward = Vector3.Cross(transform.right, up);
    }

    /// <summary>Gets Time's Delta.</summary>
    /// <param name="_pa">Time Delta's Type.</param>
    /// <returns>Time's Delta of the Given Type.</returns>
    private float GetTimeDelta(TimeDelta _timeDelta = TimeDelta.Default)
    {
        switch(_timeDelta)
        {
            case TimeDelta.Default: return Time.deltaTime;
            case TimeDelta.Fixed:   return Time.fixedDeltaTime;
            case TimeDelta.Smooth:  return Time.smoothDeltaTime;
            default:                return 0.0f;
        }
    }
}

I also made a quick Character's script for the sake of giving a quick example (the original Character script I have a has tons of dependencies). So its jump does not have cooldown, and it doesn't evaluate if it is grounded.

Simple Character Movement's Script:

using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

[RequireComponent(typeof(Rigidbody))]
public class CharacterMovement : MonoBehaviour
{
    [SerializeField] private GameplayCamera camera;     /// Gameplay's Camera.
    [Space(5f)]
    [SerializeField] private KeyCode jumpKey;           /// Jump's KeyCode.
    [SerializeField] private KeyCode displacementKey;   /// Displacement's Key.
    [SerializeField] private float displacementSpeed;   /// Displacements Speed.
    [SerializeField] private float jumpForce;           /// Jump's Force .
    [SerializeField] private ForceMode mode;            /// Jump Force' sMode.
    private Rigidbody rigidbody;                        /// Rigidbody's Component.

#region UnityMethods:
    /// <summary>CharacterMovement's instance initialization.</summary>
    private void Awake()
    {
        rigidbody = GetComponent<Rigidbody>();
    }

    /// <summary>CharacterMovement's tick at each frame.</summary>
    private void Update ()
    {
        Vector3 axes = new Vector3
        (
            Input.GetAxis("Horizontal"),
            0.0f,
            Input.GetAxis("Vertical")
        );

        if(axes.sqrMagnitude > 0.0f)
        {
            transform.rotation = Quaternion.LookRotation(axes);
            transform.Translate(transform.forward * displacementSpeed * Time.deltaTime, Space.World);
        }
        if(Input.GetKeyDown(jumpKey)) Jump();
    }
#endregion

    /// <summary>Performs Jump.</summary>
    private void Jump()
    {
        rigidbody.AddForce(Vector3.up * jumpForce, mode);
    }
}

What I Want to Know:

If I am missing something, I am using the wrong functions, calling the functions in the wrong threads/orders, etc.

Please let me know if there is more information I have to provide.

Answer 1

not enough repo for commenting yet, so writing here, because it's a common thing with jittering near the end-position (not sure if that is your problem because it's not stated in your question)

If your actual camera is approaching the end position and you're applying your next translation/rotation it may overshot by a fraction and in the next frame it does the same in the other direction. Basically shaking the image then. Normally (if you're not just lerping to the end position - which may jitter as well, because moving smaller and smaller fractions, rasterized to pixels int the end) you just have to define some "snaping threshold" which will set the position/rotation to the desired end position once that threshold is reached

Answer 2

Your GetSmoothFollowRotation() function is wrong.

SmoothDampAngle gives you an angle in degrees (0-360 or -180-180), but you're passing it as the interpolation parameter to Quaternion.Slerp that expects a unitless interpolation ratio 0-1. This is not geometrically meaningful and effectively discards any smoothness guarantees you were getting from SmoothDampAngle, allowing your camera to overshoot, backtrack to correct, and vibrate as a result.

A more correct way to use smooth damping for rotation would be to damp your look vector, then look at that:

Vector3 _lookDirection;
Vector3 _lookDirectionVelocity;

private Quaternion GetSmoothFollowRotation()
{
    _lookDirection = Vector3.SmoothDamp(
        _lookDirection,
        GetLookDirection(),
        ref _lookDirectionVelocity,
        rotationFollowDuration,
        maxRotationFollow,
        Time.deltaTime
    ).normalized;
    
    return Quaternion.LookRotation(_lookRotation) * rotationOffset;
}

Answer 3

Try updating you camera transformation in the MonoBehaviour.LateUpdate function.