The game I'm developing has a simple game server which stores inventory and hero data for each user, as well as deciding the outcome of some random events. The server interacts with PostgreSQL and responds to HTTP queries. It runs on EC2 and Microsoft Azure, but these hosts sometimes become unreachable for a few minutes, and one week the EC2 server froze every day. What are the strategies to make sure a server is always available?
Non-solution: Multiple servers with their own databases, and the client will try to connect to a different server after a timeout. Of course, their data will be absent if it's a different server.
Solution 1: As above, except whenever player data changes, that change is stored, and each player's data is replicated to the other servers (bidirectionally). The disadvantage is that this solution is "home grown" and prone to mistakes.
Solution 2: One primary server (let's leave sharding for later), and one or more backup servers, with the database being replicated. When the primary server becomes unreachable, a backup server is made into the primary server. Problems: this needs additional technology for directing queries to the right server, and also to decide when a backup server needs to be promoted to primary. Can this "manager" also hang? It sure can if it's on AWS! That would bring the whole system down.
Solution 3: We could pay for database hosting with some guarantee of reliability, and have multiple servers but just one database. We would assume the database is reliable (but still back it up). The user could connect to any reachable server, and the servers would not need to worry about having stale data, since there will be only one database. (Again, sharding is a separate problem, and we could just multiply the whole setup without fundamentally changing its architecture.)
So how is reliability achieved when individual servers aren't reliable? I like solution 3 for its simplicity, if a 3rd party database can be relied upon.