Take the 2-minute tour ×
Game Development Stack Exchange is a question and answer site for professional and independent game developers. It's 100% free, no registration required.

Okay so I am working on a game engine and this was my previous game loop:

Game::Run()
{
    While(!m_GameDone)
        Client.Update();
        LocalServer.Update();
}

but what I want to do is:

Game:Run()
{
    ClientThread = new CThread(...);
    ServerThread = new CThread(...);
    ClientThread.StartTask(Client::Run);
    ServerThread.StartTask(LocalServer::Run);
    CThreadSystem.WaitForTasksToFinish();
    return;
}

And then Client::Run and Server::Run would be:

Client::Run()
{
    while(!m_Gamedone)
    {
        this.Update();
        this.Render();
    }
}
Server::Run()
{
    while(!m_Gamedone)
        this.Update
}

So my two big questions are.. Is this a good idea? and How do I make a thread execute a function from a class instance? The threading system I am using right now is WinAPI and XTL (not STL).

UPDATE:

Okay, everyone wants to tell me about locks and data sync. The client and server threads DO NOT SHARE ANY COMMON DATA/OBJECTS. The client and server threads have their own serialized update loops and don't use any common classes or structures. I just wanted to know how to start a thread with a function in class instance scope.

share|improve this question
    
The idea seems fine to me, just be very, very careful with shared variables as you have to do with multithread programs. And this might be very hard to debug. –  Valryon Jan 24 '12 at 10:22
3  
Why are your threads called client and server? This does not look like a multi threading set up for a game to me. How are you sharing data between the treads, how do you keep then synchronised? How is the job delegation done? –  eBusiness Jan 24 '12 at 14:19
    
There is no shared data between Client and Server. The game is behaving as if they were running on separate executable. –  Matt Jensen Jan 25 '12 at 0:01

3 Answers 3

up vote 2 down vote accepted

Yes this is a good idea in general though I would consider splitting the server side out and making a dedicated server.

Secondly here is how you launch a Win32 thread and get it to call a function in a class.

class MyClass
{
     public:
     static void ThreadStartLoc(void* MyClassPtr)
     {
         ((MyClass*)MyClassPtr)->InternalThreadRun();
     }

     private:
     void InternalThreadRun()
     {
          // Do your loop here
     }
}

Start the thread with this call.

MyClass* MyClassPtr = new MyClass();
_beginthreadex(NULL, StackSize, MyClass::ThreadStartLoc, MyClassPtr am, 0, NULL);
share|improve this answer
    
Thanks, I needed the static Idea. But I can't split it into a dedicated server. –  Matt Jensen Jan 25 '12 at 0:04
    
not ALWAYS a good idea. Better do some simple thread programs before. The sync for a game may be a madness! –  Gustavo Maciel Jan 25 '12 at 1:25

Use duplicated data synchronized with message passing via 1 directional ques. It lets you scrap any kind of locking. It should also be a very flexible system.

Here are a few pointers, it's basically a system I worked out to tie in threading, multiplayer network replication and saving/loading. It's only theoretical, i've never implemented it so I cant say much about the performance.

This is a lot of text but typing it out helps me flesh it out too ☺

Firstly understand that mutex locks are going to kill performance and should be avoided. If you have 2 threads and they both lock the same game world state for their entire operation then they will be slower then just going unthreaded. Atomic stuff is better performance wise but if we get creative we can actually avoid it.

Don't base your synchronization around 'Objects' or 'Entities'. Base it around individual properties.

Give each property a unique id. Depending on your requirements, a simple game might just use an unsigned int, a more complex game like a MMORPG might use a UUID (32bytes) of which part is a timestamp that should allow ids to be generated across multiple separate servers that don't even know about each other with a statistically close to impossible chance of a id collision. If you keep those id's in sync rather than generating them dynamically this will help with network replication too (the simplest way is to transmit the whole list of dynamic properties, their current values and ids).

Named strings ids can help with scripting so you can do things like "dungeon12.dragon3.health = 1000" and/or print out a human readable list of all ids and their values for an object (which can help with things like writing level editors), but names add parsing overhead, so if you do want that it's probably best to have regular numbered ids too for things like threads and network.

Work out what properties are 'static' (or const) and what are 'dynamic'. The static stuff can be ignored with regards to synchronization. The memory could also be shared between threads without any synchronization and doesn't need to be transmitted to clients over a network (provided it's 'shipped' with your game (A custom level might need to be transmitted it the client doesn't have it though, and maybe you want to make a really flexible engine that allows you to alter vertex information on meshes which would be useful for a networked 3d editor). Things like unchanging objects in the level, the level itself, mesh information. The dynamic data will probably only be a small fraction of all your game data.

A good idea is to give each property a flag/timestamp that indicates it was changed (or created) at runtime (one flag for properties changed since file was loaded from disk, another one for properties that have changed since the last network/thread replication transmission). This allows you to make fast and compact saves since you only need to store the changes from the initial state. You can also let players jump into the middle of a game on a network and just upload a list of all the properties. It saves manually separating the properties in dynamic and static.

As a side note if you want almost instant loading/saving find a way to keep all the dynamic values separate from the static ones then when you save you can just 'snapshot' the current game state, copy it off to another block of memory and spawn a background thread to write it to the disk while the player keeps playing. Loading can just scrub the current dynamic state and restore the saved properties without effecting the unchanged ones.

Your properties are no longer going to just be basic data types in a class, they will be small classes of their own.

Look into serialization. That's the art of taking data from program classes and turing it into something that can be transmitted or stored to disk. Stuff can be serialized to bother binary, plaintext and XML. It might be handy to be able to choose between them (network and thread communication would want binary, XML might be usable for loading levels from disk and gives you an editable format).

For increased performance try to keep all the property information in a flat list somewhere so you don't have to walk up and down a tree, octree, BSP or whatever and query all the information inside of each object. It also helps with the above save/load system since you just copy the list.

Give each thread its own copy of the dynamic data, we will synchronize them manually. Ensure that 'Client' thread only has dynamic information it needs to know about, that is stuff the player can actually see or stuff with in range. In addition to reducing what we have to deal with, this also helps with preventing network cheating (the player can't wall hack if there is no entity information of what is behind the wall, of course that would require your server thread to work out what every player actually can see which is possible if you render a really low resolution low graphics version for each player but a a simple ranged/clipping frustum based system is more likely but that still prevents players from knowing about stuff on the other side of the map). You might be able to tie this into your 3D drawing code here since it will be doing similar things when working out what is visible and needs to be drawn, it's also handy to separate your render scene graph from your game world information, there's no sense in keeping mesh/texture/VBOs and so on inside your game objects, that's all render information. It also helps keep your game API agnostic when it comes time to port from DirectX to OpenGL or some kind of ASCII renderer.

Work out a property ownership/security. Just about everything will be 'owned' by the server except a few special classes for clients to send their information too. Stop clients from having the power to edit values they shouldn't but lets you use 1 global system for managing properties (ie rather than treating client information as different from the gameworld properties).

Work out the 'flow' of information. For example the 'Server' thread will be updating the 'Client' thread with regards to the entities in the game world. It won't normally be happening in reverse, the client shouldn't be updating entity information directly even on their own character otherwise it would allow hacks to teleport around the map. Instead a special controller object that they 'own' should be used that the server watches and uses to sync the game world's representation of a player.

To keep things in sync between each thread/network client use uni-directional FIFO message passing pipe RPC ques. Basically a simple FIFO that only has a single producer and a single consumer can be implemented in such a way that no mutex locks or atomic information is needed. Technically you might want a single atomic on the que itself depending on what your programming language/compiler do, it might be that all simple variable changes are atomic without having to do anything. Also depends on how 'theoretically' safe you need to be vs what actually works in the real world. C++ would probably work without issues in most places but technically the standard doesn't say anything about it so it's theoretically possible for your computer to explode, C++11 introduces atomic<>. I don't know what C# does. These FIFO ques can be used between threads and between network clients.

You can do some plumbing on your FIFO's to allow them to be split to multiple consumers or combine messages from multiple producers. (Basically just a list of FIFO's that actss like its just one big message que).

Use an Observer Pattern for your threads. Give each observer a dedicated que. You can thread threads and remote clients as basically the same just one goes across the network. Put your security in the network receiving part.

Work out a RPC language for your messages. What need to be transmitted across the network. The same can be used for inter-thread communication.

  • UINT - Message sequence number
  • BYTE - Op code
    • Set int/uint.
    • Set string.
    • Set float.
    • Set array.
    • Add object to list. (we tread a list as a property, it lets you add objects to a level). Do you create new objects here or is that separate.
    • Remove object from list.
    • Request Ping (So we can calculate lag)
    • Send Ping response (Pong).
  • UINT - Property ID
  • DATA - Property value, length depends on type. Some types will need their length appended at the start.
  • Checksum?

You could have special opcodes for things like setting the player name or a bit of text in chat, but otherwise you could just give the player ownership of special string properties that the server watches. Player set the string, server send a chatmessage/namechange request. Actually using that method you could perhaps decompose the above into something even simpler there the network codest just involve setting a few special state objects on the server. You would have SetRawDara, LengthOfData, Id. And floats, ints, strings, arrays and so on would all be treated the same way. Just leave it up to the server to check its valid (does player own that object, is the data size correct) and work out what to do with it.

You probably want to use both TCP and UDP. For things like player location UDP is fine. Things like chat or transmitting the initial state would need reliability.

For thread->thread communication you might actually want a different RPC. You actually might be able to use a list of lambada functions and ditch the protocol/parsing totally (On C++11 this should be possible, don't know about C#). Or use function pointers. If you do that network stuff would need to be treated somewhat separably but that's probably for the best. You don't need the security ownership stuff in thread->thread either and if the source is internal you can trust that the size of data being set it sane.

Each thread/client checks the que(s) before (or after) it calls its update function. If there are new messages it looks at the operation

Server thread has a Game World State. It contains every single object. When a property it owns is updated it will transmit it to all listening (or ones within range) 'client' threads via the FIFO. It will listen to a list of ques, one from each client. Its loop will basically be:

  • Check incoming thread.
  • Process incoming messages (This should be a FIFO that has combined all incoming client messages), update values as required.
  • Perform its own updates.
  • For each value updated (both from the incoming threads that need to be broadcast on to everyone else (unless you are doing p2p) or the internal update are added to a list of things to be broadcast to all clients.
  • Send a broadcast message to all listening clients.

Come to think of it, working out what player is listening to a particular 'property' might be a bit hard since a health property won't have any concept of the object itself or where it is in space so maybe ensure that each property has a parent reference.

Network thread(s). Either one global one, one per-connection, or a pool that processes network packets. This is the same for both clients and server.

  • Listen for a packet.
  • Check the validity on the packet
    • Does the client 'own' the property id it is trying to change.
    • Is the size correct.
  • If valid, convert it to the internal message representation (function pointer, lambada function, opcodes whatever). Add it to the FIFO que for this client.

Client threads:

  • Process messages from server (possibly from other clients if your doing p2p).
  • Update properties
  • If using a separate rendering thread send relevant messages to that (ie just visible information, position, any new objects in view)

Rendering thread (this can be sound too ☺):

  • Receive and process messages.
  • Render.

Physics thread is a bit more complicated. My recommendation would be to ensure that your physics engine works predictability so all clients will get an identical result. Rather than transmit a continuous stream of position updates, instead transmit the basic physics information (overall force being applied, velocity, current position) and let the clients work it out.

share|improve this answer
1  
I think this is a really great post, you bring up many good pointers. I do have a few limitations for my engine though. Only 6 threads are possible and only 64MB of RAM for game data. –  Matt Jensen Jan 25 '12 at 12:32

Assuming your Update() methods look like this:

for each entity in allEntities
{
    entity.Update()
}

You'd need to either synchronize (aka lock) the entire loop:

lock(allEntities)
{
    for each entity in allEntities
    {
        entity.Update()
    }
}

Preventing almost any multi-threading gain. The other way would be to synchronize around each and every entity update call:

for each entity in allEntities
{
    lock(entity)
    {
        entity.Update()
    }
}

Again, preventing almost any multi-threading gain and is probably much slower than just synchronizing the whole loop. Synchronizations are expensive:

Another reason to avoid having many locks in a system is the cost of entering and leaving a lock. The lightest locks use a special compare/exchange instruction to check whether the lock is taken, and if it isn't, they enter the lock in a single atomic action. Unfortunately, this special instruction is relatively expensive (typically ten to hundreds of times longer than an ordinary instruction)

http://msdn.microsoft.com/en-ca/magazine/cc163744.aspx

That said, there are multi-threaded solutions that either avoid or minimize locking. One way is to use 1 thread for the core game and other threads for non-core game elements, like generating and animating trees and animals in Flight Sim X [ref].

An other way to do it is to have 2 copies of every game entity (wasteful, I know). One copy would be the present copy, and the other would be the past copy. The present copy is strictly write only, and the past copy is strictly read only. When you go to update, you assign ranges of your entity list to as many threads as you see fit. Each thread has write-access to the present copies in the assigned range and every thread has read-access to the all past copies of the entities, and thus can update the assigned present copies using data from the past copies with no locking. Between each frame, the present copy becomes the past copy, however you want to handle the swapping of roles. [citation needed]

If you aren't noticing performance issues, I wouldn't change anything. Multi-threading issues can be tough to track down.

share|improve this answer
    
I was assuming the Update() functions were not the same function, i.e. they wouldn't be updating the same things. But if they are the same, this is a good answer. –  Byte56 Jan 24 '12 at 18:42
    
Yeah, if the Update()s are working on different sets of objects, then there shouldn't be a problem with using multiple threads. –  John McDonald Jan 24 '12 at 19:00
    
This would be useful for most people but it isn't relevant to my game. –  Matt Jensen Jan 25 '12 at 0:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.