Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

I'm developing a Real-Time game which should hold thousands of players in real-time (FPS like. max 1s lag). What would be the best infrastructure for this?

My idea was using 2 server clusters - one for the Server End (all the computing side) and one for the Database end, where a load balancer is "responsible" for each of the clusters. One main server will receive the requests from the users and send back the IP address of the relevant server that the user can work this.

The database cluster will use database replication for consistency between the databases.

There should be a geographical load balancer as well - so it will assign the regional load balancer to each user for best response.

I'm using .NET + MSSQL for the game.

Thanks!

share|improve this question
2  
When you say "infrastructure" do you mean software, hardware, services, or did you just want us to critique your existing plan as is? –  Tetrad Jan 24 '11 at 15:40
1  
@Nate - we are planing to scale up, not duplicate - so it should be fully scalable. –  roman Jan 24 '11 at 15:54
2  
-1 because teddziuba.com/2008/04/im-going-to-scale-my-foot-up-y.html - You don't cover any capacity requirements, you say the game isn't sharded but is it zoned, etc. The nature of performance optimizations - including scalability - requires specific information, and there is no absolute best architecture. –  user744 Jan 24 '11 at 17:19
1  
I would rephrase your question to something more concrete. What is 'best', in a factual, non-subjective way? Do you mean easiest to scale, easiest to manage, easiest to get going, fastest, or what? –  The Communist Duck Jan 24 '11 at 19:38
1  
"Easiest" is also subjective. Easiest for whom, over what timeframe, given what resources? Stack Exchanges work best with specific questions - "I have built this server using LINQ and MSSQL, but it fell over after 70 transaction/second, here are my top two transactions accounting for 73% of my runtime, how should I increase my throughput?" –  user744 Jan 24 '11 at 20:48

2 Answers 2

There is no best architecture without knowing a lot more about your requirements, eg. the sort of interactions between characters, how much data is going to be persistent, and so on.

If you can cope with 1 second of latency, you can probably host 1000 players on a single server without problems - but that actually conflicts with the idea of an FPS as they typically require much lower latency, eg. less than 100ms. But a system that can deal with high latency can afford to do everything via message passing which makes consistency pretty trivial. Providing your logic is quite simple, that is - complex logic gets even worse when you turn it into a message-based system rather than a locked-object system - but it all depends on your application's needs.

Similarly, if you're not persisting much data, you don't need the database machine at all, but without knowing that, it's hard to say. If you're persisting small amounts of data, and perhaps only doing it at the end of a tournament or something, again you don't need a separate database, certainly not a cluster of them. On the other hand if you're not persisting much but reading a lot, that's where the replicated databases can help you - but it also indicates that a relational database may not be the best match for your problem in the first place. Often an in-memory cache is a better solution. Similarly, if there are no transaction-style interactions between characters, then consistency becomes less important. (And if there are only a few such transactions, you can make them a special case.)

In fact, be wary of adopting an RDBMS just because it is the done thing in large systems. Although I personally do approve of using them in online games, it's best to look at your requirements and figure out your persistence strategy from that, rather than taking your preferred database and then trying to tweak it with caches and replication to get it to fit your app. You may find that all you need is offline reporting capability, in which case it's probably best to have a background process logging from your game persistence mechanism into a remote RDBMS somewhere else.

share|improve this answer

Disclaimer: my game programming experience is based around client-side single player games, but I have a background in web applications (specifically on the Microsoft stack), so that is where I'm coming from with this answer, I feel that much would apply, but without actual testing a real game server its difficult to say how it will apply, but here goes. Know this: I haven't deployed a game server, only webapps.

I would suggest a two (server) tier approach. A database tier and an "application" tier; with the third (presentation) tier being your game client.

Relational databases, are great at querying data, and decent at writing data. The key is to serialize your database writes into manageable size chunks that your cluster can handle. The more advanced editions (Data center/Enterprise) of SQL Server support clustering and replication. I would start by building a small cluster and running some queries against it to see how it works.

In the application tier, if you're doing "zoning" or something similar, you can probably get away without setting up any clusters, and simply setup a server per zone. If your zones become to big, you could setup a cluster for each zone.

You will want to build a serialization process for sending data from application tier --> database tier. The key is to have multiple levels of serialization going on. Something like this:

  • Level 1: Save to DB every X seconds, includes critical data:
    • Player Health
    • Player Items/Pickups
  • Level 2: Save to DB every 2X seconds, includes medium data:
    • Player locations
    • NPC locations
  • Level 3: Everything else, as infrequently as possible

This will keep your writes consistent and predictable, depending on the nature of your game, you could have infrequent database writes. The key is to realize that if your application server crashed, you'd have to come back online from the state in your database, so serializing player inventory every 90 minutes might make players upset.

For reading data, you'll want to load as much as possible into memory in the application tier as possible, then insure that all of your code uses this memory pool, in the background you can synchronize the memory pool with the database. As Joe points out, there will be times when you need "real-time" transactions. By serializing most of your writes, you should still have sufficient IO on your database to do real-time transactions when necessary, presuming sufficient hardware on the database server/cluster.

share|improve this answer
    
-1 because you have just thrown out ACID and are just using the database as a big data store. That's fine, Windows' disk scheduler is crappy enough that is still a performance gain, and you can probably do some neat offline metrics with the data in it, but you still need ACID - i.e. a database - backing the transactions in the game itself, in real(ish) time. –  user744 Jan 25 '11 at 0:28
    
While sparse, that is what I was going toward in the last paragraph. I'm recommending a bit of a hybrid, where you use IN-Memory where possible, and the database when necessary. –  Nate Jan 25 '11 at 5:36
    
The problem is that you have glossed over what's in memory, which is an entire other database, and you've given no direction on how to implement that, which is the actual tricky bit. –  user744 Jan 25 '11 at 10:39
    
True, though the question was about big picture architecture, not implementation details. –  Nate Jan 25 '11 at 15:54
    
True, but you've left out the entire middle tier. Is it a low-latency RDB? An ODB? Key/value store? No DB and give up on ACID? STM or locking? To be fair, it's very hard to answer that because there's not much information in the question, but all this answer does to the architecture diagram is take the big bubble in the middle with a "?" in it and connect two more services to it, not actually fill in what "?" is. –  user744 Jan 25 '11 at 19:38

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.