How to manage requests for a MMOG?

Question

I read that most servers handle more than 50 requests per second. Is this true? how many requests can a normal server handle?

How do MMOG manage requests having to update player actions instantly?

Answer 1

I read that most servers handle more than 50 requests per second. Is this true? how many requests can a normal server handle?

It's really impossible to say -- there isn't a standard definition for "normal" server, plus what many players (and even some developers without a lot of MMO experience) tend to regard as "a server" is actually a collection of both physical and logical servers that all interact to help simulate the game world. Some of them need to service a very high number of requests per second (quite more than 50), other don't so much due to their nature.

How do MMOG manage requests having to update player actions instantly?

They don't. It's impossible to round-trip from the client to server instantly; instead, client-side prediction with server authentication is generally used to hide latency and make actions appear instant. You can more the latency around, but you can't ever really get rid of it.

Answer 2

Almost all high performance servers, MMORPG or not, use thread pools and multiple queues.

A sample design might look like this:

• On Windows, use IO Completion Ports to avoid requiring a thread per connection.
• When data is received, deserialize ("read") it into a new packet object. What I do is read the packet type (1 byte) and then use a dictionary/lookup table to find the object type. Once I have the type I use reflection to create the strongly typed packet object. I call a method on the packet to "read" the data from the stream.
• The packet, along with a "network session" object representing the connection are added to a global queue. When an item is in the queue a network dispatch thread wakes up and "routes" packets to.
• The packet router checks the type of packet. If it's a generic "network" packet (for example, "Ping"), it gets wrapped in a NetworkAction object and passed to the "network action queue". If it's a "simulation" packet (ie/ "Move Player") it gets wrapped in a SimulationAction object and passed to the "simulation action queue".
• The simulation action queue works the same as the network dispatcher - once an item is in simulation action queue a simulation dispatch thread wakes up and processes each message in the queue. In other words, only one thing at a time happens to a simulation, but multiple game simulations (ie/ regions, maps) can be loaded at once. Each simulation has its own queue.
• Rather than having one thread per queue or one thread per simulation, a thread pool is used. Threads are allocated as-required.

You might want to take a look at how IIS (Microsoft's Web Server) for Windows 2008/Windows 7 is architectured internally and use a similar approach.

• One "process" (http.sys in the kernel) listens on port 80. (MMORPG - your server process listens for new requests)
• Http.sys deserializes the packets as fast as possible, handling thousands of connections at once. (MMORPG - use IO Completion Ports and thread pools to read packets)
• Once enough information is known about the request (HTTP header) it passes the information to Windows Process Activation. (MMORPG - once you've read your packet, look at the packet and determine if it needs to be sent to the "game world" or handled directly. If it's a "game world"/simulation packet, toss it in a queue so that it can be "routed" to the correct simulation)
• Windows Process Activation determines which website should process the request. If a process is not running for the website, it starts one. If no one has used the website in awhile, it stops it. If the website crashes, it restarts it. Once the website worker process is ready, it passes the message to it. (MMORPG - Don't waste resources processing areas of the game world that don't have players. Determine where the packet should go, and if its an area of the world that is offline, start it up. Pass the packet into the "simulation action queue" - one for each simulation. Use simulations to subdivide your world - for example, one per map, one per region, etc)
• The website's worker process has a queue of all outstanding requests. It spins up a number of threads in a pool (~10-20) and once a thread is available, it passes the request to the thread. The thread remains blocked until the webpage is ready (ie/ read blog listings from database and generate HTML) and returns the content back to HTTP.SYS so it can be converted to a packet. (MMORPG - difficult to have multiple threads running in the world at once due to collision detection and entity graphs, but have one thread that processes items in the queue, perhaps using a priority queue).

I use C#, mainly because it simplifies most of what I described above. It has its own internal thread pools. Strangely enough, take a look at the Microsoft Robotics Platform. It has a component called the CCR - "Concurrency and Coordination Runtime" that makes it easy to build dispatch queues and multithread easily. They also have a service component that will route messages to services and start them as required.

The key point I'm trying to make is non-game services handle hundreds to thousands of requests per second on a single core machine with minimal CPU load. Games are more complex, but if you incorporate some of the same strategies it will go a long way. Just remember - keep it simple at first, get something working, and build out the complexity only when it's required.