Quaternion.RotateTowards not working as expected when parent object moves/rotates

Question

I have a Tank with a rigidbody. A Turret Base is attached to the Tank via a Hinge Joint. The below TurretController script is attached to the Turret Base in order to rotate it to point towards the direction of the mouse.

The turret rotates flawlessly when the tank is stationary. But once I start moving the rotation behavior becomes very erratic: https://youtu.be/v1VuTbMtcCM

If anyone is able to help/explain this behavior it would be much appreciated!

/// <summary>
/// Rotates a turret towards a point at a fixed angular velocity.
/// </summary>
public class TurretController : MonoBehaviour
{
    public float m_DegreesPerSecond = 40f;

    private Vector3 m_TargetPosition;

    // Received from mouse move event
    public void Rotate(PointInputInfo inputInfo)
    {
        m_TargetPosition = inputInfo.Point;
    }

    private void RotateTurret()
    {
        Vector3 direction = Vector3.Scale(m_TargetPosition - transform.position, new Vector3(1, 0, 1));
        Quaternion targetRotation = Quaternion.LookRotation(direction, Vector3.up);
        Debug.DrawLine(transform.position, m_TargetPosition);

        // Rotate at a fixed angular velocity.
        transform.rotation = Quaternion.RotateTowards(transform.rotation, targetRotation, Time.deltaTime * m_DegreesPerSecond);
    }

    private void FixedUpdate()
    {
        RotateTurret();
    }
}

Answer 1

As DMGregory pointed out it's unwise to mix Rigidbody physics and direct transform manipulation.

Below is my solution using torque to rotate at a fixed angular velocity (in degrees per second). I apply an instant counter force when the direction is within tolerance to avoid constant adjustment jitter.

Note that for high values of m_DegreesPerSecond, the tolerance values have to be increased in order to stop jitter, which isn't ideal.

    private void RotateTurret()
    {
        // Direction of the target from our current position.
        Vector3 direction = Vector3.Scale(m_TargetPosition - transform.position, new Vector3(1, 0, 1));

        // Get the angle between transform.forward and target delta.
        float angleDiff = Vector3.Angle(transform.forward, direction);

        // Get the cross product, which is the axis of rotation to get from one vector to the other.
        Vector3 rotationForce = Vector3.Cross(transform.forward, direction);

        Debug.DrawRay(transform.position, direction, Color.magenta);

        // The difference between current velocity and the velocity I want.
        float velocityAdjustmentScalar = (m_DegreesPerSecond * Mathf.Deg2Rad) - m_Rigidbody.angularVelocity.magnitude;

        // If we're within tolerence
        if (angleDiff < 1.5 && angleDiff > -1.5)
        {
            // Apply counter torque, to prevent constant adjustment jitter.
            m_Rigidbody.AddTorque(-m_Rigidbody.angularVelocity, ForceMode.Impulse);
        }
        else
        {
            // Add torque towards the target,
            // scaled by the velocityAdjustmentScalar in order to maintain a fixed angular velocity.
            m_Rigidbody.AddTorque(rotationForce * velocityAdjustmentScalar, ForceMode.Force);
        }
    }

EDIT:

For anyone that's interested and for future reference, my final solution was to make use of the HingeJoint's JointMotor. This comes with the added advantage of being able to easily set a fixed target velocity and a maximum force to apply to maintain that velocity. For example you can mimic resistance/breaking force by setting the target velocity to 0 and the force parameter to some higher value, which provides resistance up to the maximum force.

    private void RotateTurretUsingMotor()
    {
        // Direction of the target from our current position.
        Vector3 targetDirection = Vector3.Scale(m_TargetPosition - transform.position, new Vector3(1, 0, 1));

        // Get the angle between transform.forward and target delta.
        float angleDiff = Vector3.Angle(transform.forward, targetDirection);

        // Get the cross product of the transforms forward and target direction,
        // resulting in an orthogonal vector in the direction of the axis of rotation.
        Vector3 rotationDirection = Vector3.Cross(transform.forward, targetDirection);

        // Divide the angular tolerence by 2 to represent the number of degrees tolerance.
        if (angleDiff > m_AngularTolerance / 2 || angleDiff < -m_AngularTolerance / 2)
        {
            // Add positive or negative angular velocity depending on the rotation direction.
            if (rotationDirection.y > 0)
            {
                m_JointMotor.targetVelocity = m_TargetAngularVelocity;
            }
            else
            {
                m_JointMotor.targetVelocity = -m_TargetAngularVelocity;
            }

            m_JointMotor.force = m_Force;
        }
        else
        {
            // Set the motor to maintain a target velocity of 0 up to a maximum level of force.
            m_JointMotor.force = m_Force;
            m_JointMotor.targetVelocity = 0;
        }

        // Apply the changes to the motor.
        m_HingeJoint.motor = m_JointMotor;
    }