I'm assuming you are generating a scaling matrix to perform zoom.

First is the easy part(you probably have this already). You want is to stop zooming when the background becomes the same size as the viewport. The point to stop zooming out is 'backgroundwidth/viewportwidth'. This will be 0.5 in your case. Don't let the zoom factor go below this value.

This  will work fine if the camera is in the centre of the background, but if the camera is off centre, you will still see your cornflower blue on the edge of the screen.

In order to prevent the user from being shown outside the edge of the world, yo will need to change the size of `playBounds` rectangle. You are already using this to stop camera movement when the player gets too close to the edge. You just need to make sure that this rectangle has the appropriate size. Basically you want to prevent the camera position from getting within `camerawidth/2` pixels of the edge of the background. During zooming out the background is effectively getting smaller

    float scaledWorldHalfWidth=worldSize.x*cam.Zoom/2.0f;
    float scaledWorldHalfHeight=worldSize.y*cam.Zoom/2.0f;

    playerBounds = new Rectangle(worldcentre.X-scaledWorldHalfWidth +CameraViewport.X/2,
                                 worldcentre.Y-scaledWorldHalfHeight+CameraViewport.Y/2,
                                 worldcentre.X+scaledWorldHalfWidth -CameraViewport.X/2,
                                 worldcentre.Y+scaledWorldHalfHeight -CameraViewport.Y/2);

In this code I am  decreasing the size of the rectangle as the zoom factor decreases (as the player sees more and more of the background).