What you're looking for is an ArcBall Camera. I've got a full snippet over here https://roy-t.nl/2010/02/21/xna-simple-arcballcamera but just to explain the general idea:

You set a look-at point in space which you want to orbit. Then you create a vector from that point by rotate around it using the Pitch Yaw and Roll. You then lengthen the vector to your desired distance from the object. 

Anyway the most interesting stuff happens in this method:

    private void ReCreateViewMatrix()
            {
                //Calculate the relative position of the camera
                position = Vector3.Transform(Vector3.Backward, Matrix.CreateFromYawPitchRoll(yaw, pitch, 0));
                //Convert the relative position to the absolute position
                position *= zoom;
                position += lookAt;
     
                //Calculate a new viewmatrix
                viewMatrix = Matrix.CreateLookAt(position, lookAt, Vector3.Up);
                viewMatrixDirty = false;
            }