Or I can get it moving on the x but if the player walks into tiles from the side the character won't respond.

Well, I've noticed one fallacy in your logic that is most probably what's leading to this:

    for (int i = 0; i < tileList.Count; i++)
        {
            if (potentialNewPosition.Intersects(tileList[i].GetRectangle))
            {
                position = oldPosition;
                colliding = true;
            }
        }

If a collision is found, the player's movement is simply 'cancelled'. Here, if he collides with a block, he simply is sent back to where he came from. What we should do when dealing with tile collision is not cancel a movement, sending the object back to its starting position, but rather correct it.

What your algorithm should do is place your player's hitbox outside of the colliding tile, but no further from it. For example, considering that your player is basically a rectangle, if it was found to be inside a tile, you could make it so that Player.Right = Tile.X (if he was moving rightwards), or Player.Bottom = Tile.Y (for downward movement), or Player.X = Tile.Right (leftward), or Player.Y = Tile.Bottom (upward). This places him right alongside the tile, outside.

Or I can get it moving on the x but if the player walks into tiles from the side the character won't respond.

Well, I've noticed one fallacy in your logic that is most probably what's leading to this:

for (int i = 0; i < tileList.Count; i++)
{
    if (potentialNewPosition.Intersects(tileList[i].GetRectangle))
    {
        position = oldPosition;
        colliding = true;
    }
}

If a collision is found, the player's movement is simply 'cancelled'. Here, if he collides with a block, he simply is sent back to where he came from. What we should do when dealing with tile collision is not cancel a movement, sending the object back to its starting position, but rather correct it.

What your algorithm should do is place your player's hitbox outside of the colliding tile, but no further from it. For example, considering that your player is basically a rectangle, if it was found to be inside a tile, you could make it so that Player.Right = Tile.X (if he was moving rightwards), or Player.Bottom = Tile.Y (for downward movement), or Player.X = Tile.Right (leftward), or Player.Y = Tile.Bottom (upward). This places him right alongside the tile, outside.