I'm assuming you are generating a scaling matrix to perform zoom.
First is the easy part(you probably have this already). You want is to stop zooming when the background becomes the same size as the viewport. The point to stop zooming out is backgroundwidth/viewportwidth. This will be 0.5 in your case. Don't let the zoom factor go below this value.
This will work fine if the camera is in the centre of the background, but if the camera is off centre, you will still see your cornflower blue on the edge of the screen.
In order to prevent the user from being shown outside the edge of the world, yo will need to change the size of playBound rectangle. You are already using this to stop camera movement when the player gets too close to the edge. You just need to make sure that this rectangle has the appropriate size.