Handedness is referred to as 'chirality'.
Moving the camera "left" is the same as moving the world "right". The ViewMatrix can actually be constructed by inverting the world matrix you'd construct to draw a piece of geometry representing the camera. Applying the ViewMatrix, literally, un-moves, then un-rotates the world to where the camera becomes (0,0,0) and whatever direction the camera is facing becomes "into the scene".
The chirality of the ViewMatrix simply determines whether the direction of "into the scene" refers to (0,0,1) or (0,0,-1). With either chirality, (1,0,0) is "right" and (0,1,0) is "up".
I refer to this diagram whenever I've confused myself:
Notice that while both arms are pointing the same direction, the thumbs, both pointing in the Z+ direction, are opposite.
I agree with user75844; that's just how the math works out.