How to stop object with joints flying away at high velocity on collision?

Question

I am currently making a game where the player constructs vehicles out of blocks. I am achieving this through having rigidbodies for each 'group of blocks' separated by 'function blocks' (e.g. motors, that can rotate) to allow for movement of the rigidbodies/groups of blocks. These function blocks connect the rigidbodies by HingeJoints.

I added gravity as the game will be set in space. I update the velocity of the vehicle in a method called on a fixed timeStep of 0.01, using AddForce.

private void UpdateVelocity()
{
    rb.AddForce(velocity - lastVelocity, ForceMode.VelocityChange)
}

The problem is, now whenever I collide with something, the vehicle freaks out and flies away at ridiculously high velocities. I've tried many different approaches but I can't figure out the solution. I don't think it's related to me using VelocityChange as it doesn't work with other ForceModes either.

I am relatively new to using joints in Unity so any help would be appreciated!

edit:

You should be able to reproduce with this:

GravityBody is a script attached to every object that should interact with gravity (eg. a planet, a vehicle, etc.)

public class GravityBody : MonoBehaviour
{
    public Vector3 initialVelocity;
    public Vector3 Velocity { get; private set; }
    private Vector3 lastVelocity;


    private Rigidbody rb;
    public float Mass { get; private set; }


    private void Awake()
    {
        SpaceSimulation.gravityBodies.Add(this);



        //configure rigidbody
        rb = GetComponent<Rigidbody>();
        rb.useGravity = false;
        rb.isKinematic = false;
        rb.centerOfMass = Vector3.zero;
        rb.interpolation = RigidbodyInterpolation.Interpolate;
        rb.collisionDetectionMode = CollisionDetectionMode.Discrete;


        Velocity = initialVelocity;
    }



    public void UpdateVelocity(Vector3 acceleration, float timeStep)
    {
        Velocity += acceleration * timeStep;
    }

    public void UpdatePosition()//float timeStep)
    {
        //rb.MovePosition(rb.position + Velocity * timeStep);       //doesn't hvae collision, below should achieve the same result
        rb.AddForce(Velocity - lastVelocity, ForceMode.VelocityChange);

        lastVelocity = Velocity;
    }


    public Vector3 RbPosition
    {
        get
        {
            return rb.position;
        }
    }
}

SpaceSimulation should be on only one object in the scene, it manages gravity interactions.

public class SpaceSimulation : MonoBehaviour
{
    //gravity
    public static float gravitationalConstant = 0.0001f;
    public static float timeStep = 0.01f;

    public static List<GravityBody> gravityBodies = new();



    private void Awake()
    {
        Time.fixedDeltaTime = timeStep;
    }


    private void FixedUpdate()
    {
        if (ModeManager.gameMode == GameMode.Play)
        {
            for (int i = 0; i < gravityBodies.Count; i++)
            {
                Vector3 acceleration = CalculateAcceleration(gravityBodies[i].RbPosition, gravityBodies[i]);

                gravityBodies[i].UpdateVelocity(acceleration, timeStep);
            }

            for (int i = 0; i < gravityBodies.Count; i++)
            {
                gravityBodies[i].UpdatePosition();
            }
        }
    }


    public static Vector3 CalculateAcceleration(Vector3 testBodyPos, GravityBody testBody = null)
    {
        Vector3 acceleration = Vector3.zero;

        foreach (var body in gravityBodies)
        {
            if (body != testBody && body.RbPosition != testBodyPos)
            {
                float sqrDistance = (body.RbPosition - testBodyPos).sqrMagnitude;
                Vector3 forceDirection = (body.RbPosition - testBodyPos).normalized;

                acceleration += body.Mass * timeStep * forceDirection / sqrDistance;
            }
        }

        return acceleration;
    }