"Is it wrong that I am counting total rotation and then creating [a] quaternion from it?"
Yes, because rotations do not combine like simple addition.
(In mathematical terms, rotations in three dimensions are not commutative)
Here's an example you can do with any old mug. Turn it 90 degrees on the vertical axis, then flip it over away from you (finish drinking your coffee before this step), then turn it another 90 degrees on the vertical. Hypothetically, you've now turned it 180 degrees on the vertical axis and 180 degrees on the horizontal, right?
But if you do all 180 degrees of vertical axis rotation at once and then the horizontal, you don't get the same result.
So, you can't model a rotation as a running total of each incremental rotation step's angles. As soon as you have more than one axis in play, the result of the "total" rotation will tend to diverge from your intuitive expectations for each incremental step. This is most pronounced in gimbal lock, when two of your incremental rotation axes become the same, but it creeps in throughout.
The solution is to separate the concepts of your current orientation from the rotation you want to apply to it.
You can store your current orientation as a quaternion or a matrix (though quaternions have some additional nice properties like being compact and less prone to accumulated rounding errors / easier to correct).
Then compute the change in rotation you want to apply as a separate quaternion or matrix.
Finally, multiply these two together to update your current orientation, getting the result of "starting from this orientation, apply this rotation at the end":
newOrientation = rotationChange * currentOrientation;
(Order matters here too: assuming you're using a multiplication convention where vectors go on the right, this says "apply the rotation change relative to worldspace, after currentOrientation has been applied" - flipping the multiplication order says "apply the rotation change relative to the object's local axes / before currentOrientation has been applied")
There are a couple of ways to apply this strategy to your case:
Fix #1: Accumulate changes every frame
Each frame, you calculate an incremental rotation from the mouse movement since the previous frame, and construct a new quaternion from these angles. Because the angles aren't being totalled over several frames they stay small and (mostly) well-behaved. Multiply this change by your currentOrientation to update it. Each incremental rotation step is performed relative to the current displayed orientation of the object.
Fix #2: Compute rotation for the net mouse movement.
Another approach is to store the mouse position and object orientation at the beginning of the rotation move (eg. when the user presses the mouse button over your object), and then each frame compute a rotation that takes the original mouse position to the new mouse position. Compose this with your currentOrientation, but save it to a new variable that you use for display only, keeping currentOrientation the same to repeat the process next frame. Only at the end of the rotation move (eg. when the user releases the mouse button) do you save the composed display rotation into the current object orientation variable. This can reduce drift from summing up many small/noisy cursor movements, which can make it fussy to get back to the original orientation.
Note that neither of these approaches is typically suitable out of the box for controlling a camera, since they don't give any special meaning to the up direction out of the box. Without some up correction, a camera will tend to roll about its viewing axis over time - see the "mostly" call-out above.