You would only restrict movement in that direction. In other words, let the player move left only he has not collided with a wall to his left. Let him move right only if there is no wall to his right. This way, if there is a wall on one side, the player can still back away.
You would do this by checking multiple collision points, so instead of box.x, you would check both box.x+(box.width/2) and box.x-(box.width/2).