Multi-threaded MMO Server with complex inter-dependencies.

Question

I am working on a Mobile MMO and I am using .NET to build a MMO Server. My current application architecture is as follows.

• Clients are accepted using Async Calls (BeginAccept/EndAccept).
• Data incoming from the client is read using Async Calls again (BeginRead/EndRead).
• The data is parsed and handled directly on the Threadpool (EndRead) call.
• I have a dedicated thread for Monsters, which is essentially a thread with a tight while loop.

This simple architecture works fine for the most part. Now the tricky part is when I try to include interactive elements. Consider the following use-cases.

• A Player attempts to attack other player B. In this case the Player A's thread will try to reduce the HP of Player B. Player B on the other hand can increase the HP using Potions. There are certain kinds of attacks which will inflict self-injury on the attacking player. In this case to reduce A's HP I will have to either have a nested lock on both the players or first reduce A's HP with a lock on A and then reduce B's HP with a lock on B. Having a nested lock will be a disaster for deadlock since if B executes the same actions, then the lock order will be reversed creating a deadlock between A and B. If I first lock on A and then lock on B, what will happen if the lock on A reduces A's HP and lock on B waits because B is attempting to teleport back to town? In this case B will take the hit when B is back in the town which is not something that should happen. I could add a check to detect the range, but if the range check fails, how will I reverse the A's HP decrease? To make things more complex, multiple players can attack A or B or each other.
• Player Discovery. If a player moves in the game, the thread of the player scans the map cells for other players. The thread would then send the appear packet of itself to B and B's appear packet to itself. This has to be done with a lock on B, since B's appearance can change with the items worn or potion used. I will have to resort to nested locks again because I will have to make sure that when I send the packet to B, A's position is fixed and vise-versa since there are skills that could affect the location of players (Pushing the player away). In case I don't lock on A, if another player tries to use a skill to push A to another position, the updated location packet of A could reach B first before A's appear packet, which will cause the latest location packet to go useless.
• Trading. Players can do 1-1 trade with other players. So when the trade happens, Player A's trade items will be transferred to player B and Player B's trade items will be transferred to player A. I will have to do something like (lock A) {(lock B) { A = BItem; B=AItem; }}. This nested lock is again dangerous. If I do (lock A){ A = BItem; }; (lock B) { B = AItem; } Imagine what would happen if the A lock section is executed and by the time B is being executed, B player gets disconnected? I will have to rollback A with another lock again which is seemingly complex.
• Monsters Attacking Players / Monster Discovery. Similar to the discussed issues with Player Discovery and Player Attacks.

Keeping the above points in mind I would like few clarifications.

1. As you can see there is a lot of inter-dependency which will only get more and more complex once there are more features introduced in the game, is threading really worth it? With the amount of obvious lock requirements for concurrency, I might as well switch everything to a single thread and not waste time on trying to fix bugs that I can't clearly reproduce.
2. Will it be a better approach to have the packet parsing and decryption on the Threadpool thread and queue all of the packets to a mainthread that will run sequentially? The monster thread could also queue character related operations like attacking to the main thread so that every action is carried out sequentially. The network calls will still be async.

Is there any better and a proven approach that I can take to deal with these issues? The key to a successful project is always a solid architecture and I want to get it right before moving into the core development.

Answer 1

I'm just a hobby gamedev but it sounds to me that you have an general desing issue.

I belive it's not a good design to handle player actions on a per thread basis, for sure you can move calculations to some threads. But in general your server just:

• receives messages
• verify them
• store them in a list
• do some calculation based on received messages
• create new outgoing messages to the involved parties

In this case just incomming and outgoing messages are handled by threads. The main gamelogic stays in you game loop which works on the list of received messages. And this would be the place where things have to be synchronized.

For example you could have som threads who received messages and write to an incomming Queue. This incommig Queue will be copied everytime your game steps ahead into maybe an MessagesToProcessQueue which you gameloop works an. As an result a OutputMessageQueue will be created which will held the information which client has to be informed about what. And your threads are just dequeueing items from here to tell the clients.

So in a multiplayer game your clients (in general) are stupid and just rendering things.

1) A Player attempts to attack other player B

For your example the server has the information about position and health for client A and B.

When A tries to attack B, there's just a message from client A to server which tells the server that A tries to attack B.

The server then does some checks:

• is player A near to B?
• was the attack successful?
• if yes, calculate health for client B
• send a health update message for client B to client A and B

2) Player Discovery

The server knows all player positions. So a naive approch would be to check at a given time for each client which other clients are inside his view range. Just check if the position is inside a circle (you define how big it is) around the player.

3) Trading

This is the most difficult thing. Because a lot of things can go wrong, and result often in bugs like item duping in games. For example a client can abort the trade. The connection can abort and so on.

So you have to create a transaction around the whole process, and when this transaction would succeed inform all parties that it could be executed and then execute it. But you must be able to roll back the whole thing if something goes wrong (Db error, network error).

You first have to check if it could be executed because you have to check things like:

• Are the involved parties near to each other?
• Does the player own the item he want's to trade?
• Does the player have enough credits?
• Many more things to check

Because even when the player owns the item at the start of the trade. He could maybe destroy it or throw it to the ground etc.

4) Monsters Attacking Players / Monster Discovery

Same as 1)