3
\$\begingroup\$

From my estimates the MMO I'm designing can support 500000 concurrently connected non-idle users (and still be able to you know, do stuff) for about $1700/month in total hosting costs (includes bandwidth). No system admin is needed as the infrastructure auto-scales. Each additional 40000 users costs another $136/month. I've spent several months designing this backend and I'm curious how much other such systems cost.

I should note that this does NOT include the cost of any assets used by the client game. Assume it's an iPhone game with bundled assets.

Thanks!

\$\endgroup\$
15
  • 10
    \$\begingroup\$ "500,000 Concurrent Connections", "No system admin is needed as the infastructure auto-scales" meet, Murphy's Law \$\endgroup\$
    – Jesse Dorsey
    Commented Jun 4, 2011 at 3:05
  • 1
    \$\begingroup\$ What are you estimates on bandwidth consumption, sustained and total GBs per month? What speed of connection will you have, 10, Mbps FDX (typical for co-location), 100 Mbps FDX, 1 Gbps FDX, or perhaps a combination of multiple 100 Mbps FDX connections? How many competing backbone providers should your co-location provider be connected to? Will you be installing equipment you own (like I do with all my co-located servers), or will you be renting equipment or virtualized instances? Which Operating System(s) will you be using (I use NetBSD Unix)? \$\endgroup\$
    – Randolf Richardson
    Commented Jun 4, 2011 at 3:28
  • 1
    \$\begingroup\$ If your estimates show that you can support half a million concurrent users for $1700/month, then either your estimates are way off, or you're about to revolutionize the intertubes (in which case I don't think that you'd be asking this question), or your MMO is highly simplistic (even then, half a million concurrent users is quite a lot). \$\endgroup\$
    – Olhovsky
    Commented Jun 4, 2011 at 4:39
  • 2
    \$\begingroup\$ I seriously doubt the "no system admin is needed" part. Who is gonna fix issues, there are gonna be bugs to be identified and patches need to be installed and tested. Who is gonna watch out for security issues? Look what happened to sony recently. It sounds a bit like a calculation for a perfect world ;) \$\endgroup\$
    – Maik Semder
    Commented Jun 4, 2011 at 8:11
  • 3
    \$\begingroup\$ @Bart they fix bugs in your own game specific software? They care when your game crashes? There must be a person that takes care of maintaining and observing your game specific code running on that servers. Whether you call him admin or not, he usually wants a paybill :) \$\endgroup\$
    – Maik Semder
    Commented Jun 4, 2011 at 12:03

2 Answers 2

Reset to default
8
\$\begingroup\$

There's too many undefined or barely defined terms there. MMO is such a loaded term. Without knowing how much CPU processing a user needs, how much backend storage a user needs, etc... It's impossible to say.

I mean just at a bare minimum assuming the hardware can support it, you would need 8 front end servers just to support 500,000 connections. (TCP max of 65,536 ports per VPS rounded down to 65k.) I assume you've read about the C10k problem (The difficulty in scaling a single server to 10,000 connections, and 8 front end servers assumes you're at 6 times that rate.) If not take a look at http://www.kegel.com/c10k.html

However, the metric I've used when it comes to MMO buildout was to assume 1 'CPU' per 100 connected users. This worked out to be a good average from 2001-2005 but I haven't developed an MMO since. This 1 CPU basically just encapsulates the cost of gameplay servers, front end servers, database servers, content distribution servers etc... It worked whether it was a blade server with 10 CPUs or old dual Pentium Pros. Of course as an added example those build-outs usually assumed one full time Network Operations person per 5,000 connected users. (Both obviously scaling with the peak-tie ratio.)

I'd advise getting to your scalability tests sooner than later because I think it's going to wake you up to some problems in your design. Get 50,000 people on 1/10th the hardware you expect to run in production and see where the seems come apart. (Ideally do that 12-18 months before you expect to ship.)

\$\endgroup\$
6
  • 1
    \$\begingroup\$ (In addition to "MMO being such a loaded term," it's also a regularly "looted" term.) ;-D \$\endgroup\$
    – Randolf Richardson
    Commented Jun 4, 2011 at 14:32
  • \$\begingroup\$ If you're running into a maximum limit of 64k ports, you can always assign multiple IP addresses to the same NIC or install additional NICs for this (Intel's server NICs are really good for this as they have a few models that supply multiple ethernet ports from a single PCI add-in card). But this ignores the need for greater processing power to deal with all this information too, so you do make an excellent point there. +1 for you. \$\endgroup\$
    – Randolf Richardson
    Commented Jun 4, 2011 at 14:34
  • \$\begingroup\$ 1 CPU per 100 users seems to be Parkinson's Law in action more than anything else. You simply can't go higher than that without server costs eating a disproportionally large amount of the revenue. \$\endgroup\$
    – aaaaaaaaaaaa
    Commented Jun 4, 2011 at 18:44
  • 3
    \$\begingroup\$ LOL. You can have more than 64K inbound connections per host. You are referring to the ephemeral port range for outbound connections which is usually setup for the range 1024-65535. \$\endgroup\$
    – z8000
    Commented Jun 5, 2011 at 18:53
  • 1
    \$\begingroup\$ A TCP connection is defined by source ip, source port, target ip, target port. So there is no 64k limit based on the port range. But a common operating system cannot handle so many connection without a lot of tweeking. \$\endgroup\$
    – Hendrik Brummermann
    Commented Jun 6, 2011 at 16:22
-1
\$\begingroup\$

Stop thinking in terms of servers and start thinking in terms of cloud computing. Build it with the Google App Engine for example. It will automatically Scale to handle any traffic growth. Besides not stressing about CPUs, servers, databases, RAM and diskspace, you also pay for just what you use.

\$\endgroup\$
5
  • 2
    \$\begingroup\$ -1 for cloud addiction. (Read lwn.net/Articles/443775, for example.) \$\endgroup\$
    – user744
    Commented Jun 6, 2011 at 8:09
  • \$\begingroup\$ Uh no. At the very least GAE is http only. \$\endgroup\$
    – z8000
    Commented Jun 6, 2011 at 14:58
  • \$\begingroup\$ Ah, since he didn't state what kind of MMO this is it might be quite feasible. There are a fair number of web/ajax driven games out there. \$\endgroup\$
    – Tim Holt
    Commented Jun 6, 2011 at 21:23
  • \$\begingroup\$ So is Mafia Wars, which is what you explicitly compared it to. Also GAE can also talk XMPP opening up a whole range of bidirectional communication. \$\endgroup\$
    – Doug-W
    Commented Jun 7, 2011 at 14:13
  • \$\begingroup\$ good points! :p \$\endgroup\$
    – z8000
    Commented Jun 10, 2011 at 2:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .